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PREFACE 


This  document  was  prepared  by  the  Institute  for  Defense  Analyses  (IDA)  under  the 
task  order.  Application  of  Distributed  Manned  Simulation  to  the  DoD  Acquisition  Process, 
for  the  Advanced  Research  Projects  Agency  (ARPA).  It  fulfills  the  objectives  of  the  task, 
to  identify  the  nature  of  the  support  simulation  can  provide  to  the  various  phases  of  the 
acquisition  process  and  to  characterize  the  general  simulation  environments  and  associated 
methods  for  applying  them. 

The  following  IDA  research  staff  members  were  reviewers  of  this  document;  Mr. 
Thomas  P.  Christie,  Dr.  Harlow  Freitag,  Dr.  Richard  J.  Ivanetich,  Mr.  Christopher  Jehn,  and 
Dr.  Karen  J.  Richter. 
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EXECUTIVE  SUMMARY 


The  Department  of  Defense  (DoD)  acquisition  process  involves  many  functional 
communities,  each  with  its  own  responsibilities  for  and  perspectives  on  proposed  new  pro¬ 
grams.  These  functional  communities  include: 

•  Requirements  and  Concept  Development 

•  Engineering  Design  and  Manufacturing 

•  Test  and  Evaluation 

•  Logistics  and  Support 

•  Force  Planning  and  Budgeting 

•  Program  Management 

Each  community  performs  its  own  trade-off  analyses  on  proposed  new  system  designs, 
and  interacts  in  successive  iterations  of  the  design  and  needed  supportive  processes  as  the 
design  converges  toward  that  of  a  final  production  item. 

This  design-convergence  process  has  parallels  in  the  commercial  world,  where 
product  development  in  large  corporations  also  involves  the  interaction  of  powerful  func¬ 
tional  communities.  Some  of  the  most  progressive  companies  have  substantially  reduced 
the  time  and  cost — by  up  to  50  percent — of  developing  new  products  by  incorporating  the 
latest  in  design,  engineering,  manufacturing  and  support  technologies,  and  management 
practices.  Such  companies,  notably  in  the  automotive,  aeronautical,  and  electronics  sectors, 
are  capitalizing  on  advances  in  three  areas: 

•  New  management  practices,  including  integrated  product  and  process  develop¬ 
ment  (IPPD)  and  total  quality  management  (TQM); 

•  Improved  design  and  manufacturing  technologies,  including  computer-aided 
design  (CAD)  tools  and  flexible  manufacturing  systems;  and 

•  The  increased  power  of  distributed  communications  networks  linking  dispersed 
elements  of  design  teams. 
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Simulation  has  aided  many  companies  in  implementing  these  advances.  It  entails 
the  ability  to  represent  the  properties  and  performance  of  systems.  Simulation  offers  four 
capabilities  that  can  support  timely  and  effective  communication,  and  thus  help  to  integrate  ( 

functional  communities  into  a  unified  program  design  team.  First,  simulation  can  be  used 
to  provide  visualization  of  the  product,  which  enhances  communication  both  within  and 
among  functional  communities.  Second,  it  permits  performance  modeling,  augmenting  tra¬ 
ditional  analytical  tools  in  creating  and  assessing  design  alternatives.  Third,  when  coupled  ( 

with  distributed  communications  capabilities,  simulation  tools  can  provide  design  team 
members  access  to  a  common  information  base  on  the  product  and  its  performance.  Finally, 
user  interaction  allows  rapid  exchange  of  information,  ideas,  and  concerns  across  both  geo¬ 
graphic  and  functional  boundaries.  < 

Defense  acquisition  programs  could  benefit  significantly  if  these  kinds  of  simula¬ 
tion  capabilities  could  be  applied  in  conjunction  with  new  management  practices  to  break 
down  the  traditional  “stovepipe”  organizational  barriers  that  have  inhibited  coordination 
and  communication  in  developmental  programs.  Such  an  acquisition  environment  would 
cut  the  time  and  cost  of  weapon  development  programs  and  significantly  improve  the  man¬ 
agement  of  program  risk.  The  challenge  is  to  implement  both  the  technological  advances 
in  simulation  needed  to  integrate  defense  development  teams,  and  the  management  process 
changes  needed  to  create  a  functionally  integrated  system  acquisition  process. 

This  study  examined  the  current  state  of  the  art  in  applying  simulation  in  defense 
acquisition  programs  to  better  understand  the  kinds  of  technical  advances  that  will  be  need¬ 
ed.  Our  examples  illustrate  both  the  wide  ranging  application  of  simulation  and  the  limita¬ 
tions  in  simulation  capabilities.  Simulation  is  currently  being  used  within  each  of  DoD’s 
functional  communities,  and  each  community  is  expanding  the  modeling  and  simulation 
tools  needed  to  perform  its  own  development  tasks.  As  yet,  however,  there  has  been  little 
incentive  for  establishing  common  tools,  data  bases,  validation  methods,  or  data  exchange 
standards  across  communities.  This  lack  of  commonality  in  tools  and  data  has  limited  the 
role  of  simulation  as  a  medium  of  communication  across  functional  communities.  Simula¬ 
tion  will  increasingly  play  this  role,  however,  as  capabilities  advance.  Three  levels  of  capa¬ 
bility  will  mark  that  evolution  of  simulation: 

•  Functional  distributed  simulation:  Provides  visualization  and  performance 
modeling  within  a  functional  community.  Most  existing  simulations  are  of  this 
type,  and  are  widely  used  within  both  government  and  contractor  portions  of 
each  functional  community. 
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•  Cross-Junctional  distributed  simulation:  Provides  common  data  and  user  inter¬ 
action  across  functional  boundaries.  The  limited  capabilities  that  exist  today  lie 
in  the  use  of  combat  simulators  by  the  requirements,  engineering,  testing,  and 
force  planning  communities.  Data  standardization  activities,  such  as  the  Contin¬ 
uous  Acquisition  and  Life-cycle  Support  (CALS)  (previously  known  as  the 
Computer-Aided  Acquisition  and  Logistics  Support)  effort  and  the  Product 
Data  Exchange  Standard  (PDES),  are  attempting  to  remove  barriers  between 
the  engineering  and  logistics  communities. 

•  Functionally  integrated  simulation:  Represents  the  long-range  vision  for  me 
evolution  of  simulation,  in  which  distributed  simulation  would  eliminate  the 
language  barriers  between  functional  communities.  Each  community  could 
access  a  common  representation  of  a  design  from  a  distributed  enterprise  net¬ 
work,  visualize  it  in  the  manner  most  useful  for  that  community’s  work,  and  per¬ 
form  needed  analyses  using  the  conununity’s  customary  analytical  tools. 

The  main  recommendation  of  this  study  is  that  DoD  establish  a  simulation  invest¬ 
ment  strategy  incorporating  the  long-term  goal  of  creating  a  functionally  integrated  acqui¬ 
sition  process.  Creating  the  needed  simulation  capabilities  will  require  investments  in 
several  basic  simulation  building  blocks:  general  simulation  hardware  and  software; 
defense-specific  application  tools;  communication  standards;  behavioral  methods  and  data; 
verification,  validation,  and  accreditation  (W&A);  and  education  and  training.  The  DoD 
simulation  community  can  most  effectively  emphasize  initiatives  and  investments  in  those 
areas  that  will  not  be  supported  as  a  matter  of  course  by  commercial  developers  or  the  DoD 
functional  communities.  These  needed  initiatives  and  investments  include: 

•  Designing  a  simulation  architecture  for  a  functionally  integrated  acquisition 
process; 

•  Preparing  a  user  guide  to  current  simulation  tools,  data,  and  applications  to 
encourage  the  sharing  of  available  capabilities; 

•  Developing  cost  and  schedule  simulation  tools  showing  how  advances  in  man¬ 
agement  pract’''“s  and  development  technologies  can  be  incorporated  in  struc¬ 
turing  'jew  progr.i  s; 

•  Developing  distributed  simulation  security  so  classified,  proprietary,  and  per¬ 
sonnel  sensitive  data  can  be  readily  exchanged;  and 
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•  Investing  in  several  specific  simulation  infrastructure  areas  which  will  directly 
benefit  acquisition  programs; 

-  Build  accessible  data  base  libraries  that  capture  prior  experience  and  help 
prevent  reoccurrence  of  mistakes; 

Improve  simulation  development  tools  and  methods,  including  those  for 
VV&A,  variable  fidelity,  and  aggregation/deaggregation;  and 

-  Establish  widely  accepted  standards  to  facilitate  development  and 
exchanges  of  models  and  simulations  within  and  across  functional  commu¬ 
nities. 

In  addition  to  these  initiatives  that  relate  directly  to  simulation,  appropriate  incen¬ 
tives  need  to  be  defined  for  organizations  both  within  and  supporting  the  DoD  to  help  en¬ 
courage  them  to  adopt  the  key  management  principles  which  commercial  industries  have 
found  necessary  to  attain  a  functionally  integrated  acquisition  process.  These  principles 
are: 

•  Selecting  strong  program  development  leadership, 

•  Using  functionally  integrated  development  teams, 

•  Establishing  clear  communications  and  expectations, 

•  Conducting  simultaneous  development,  and 

•  Implementing  effective  risk  management. 

This  study  provides  the  vision  of  a  functionally  integrated  acquisition  process  based 
on  the  use  of  distributed  simulation,  and  serves  as  a  point  of  departure  in  creating  a  simu¬ 
lation  investment  strategy.  This  strategy  should: 

•  Incorporate  existing  and  ongoing  advances  and  activities  in  the  commercial 
world,  each  of  the  defense  functional  communities,  and  the  suppliers  to  DoD. 

•  Assess  the  costs  and  benefits  of  the  selected  investment  options. 

Building  this  investment  strategy  will  require  active  involvement  of  senior  Office  of  the 
Secretary  of  Defense  (OSD)  leadership  along  with  the  participation  of  each  of  the  DoD 
functional  communities  and  DoD  suppliers  that  support  acquisition  programs. 
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I.  INTRODUCTION 


Advances  in  management  practices  and  technologies  are  revolutionizing  the  way 
U.S.  products  are  developed  and  produced.  Some  firms  can  now  cut  up  to  50  percent  off  the 
time  required  to  develop  and  field  new  products  while  improving  product  quality  and 
reducing  costs.  These  advances  have  been  made  possible  by  developments  in  three  areas; 

•  The  diffusion  of  new  management  approaches  for  design  and  manufacturing, 
which  includes  total  quality  management  (TQM),  enterprise  integration,  and 
integrated  product  and  process  development  (IPPD). 

•  Rapid  improvements  in  design  and  manufacturing  tools,  such  as  computer-aid¬ 
ed  design  (CAD),  computer-aided  manufacturing  (CAM),  and  computer  simu¬ 
lation. 

•  The  growing  ability  to  integrate  these  tools  within  distributed  computer  net¬ 
works. 

As  more  firms  adopt  these  highly  complementary  developments,  the  resulting  benefits  will 
reduce  development  and  production  time,  improve  quality,  and  reduce  costs  in  companies. 

Many  within  the  Department  of  Defense  (DoD)  and  its  suppliers  are  also  seeking  to 
adopt  these  advances  by  adapting  such  key  commercial  practices  and  technology  advances 
to  DoD  management,  organizational  structure,  and  program  developments.  One  important 
goal  central  to  those  advances  is  to  create  a  functionally  integrated  system  acquisition  pro¬ 
cess  which  breaks  down  the  traditional  “stovepipe”  or  functional^  barriers  that  inhibit  coor¬ 
dination  and  communication  in  development  programs.  This  paper  focuses  on  the  role 


*  These  include  requirements  and  concept  development,  engineering  design  and  manufacturing,  test  and 
evaluation,  logistics  and  support  force  and  budget  planning,  and  program  management.  The  activities 
within  these  functional  communities  would  include  all  events  within  a  program’s  life  cycle,  i.e.,  “from  lust 
(the  initial  idea  about  the  program)  to  dust  (the  results  of  the  disposal).”  Examples  of  such  activities  (in 
addition  to  those  implied  by  the  function  title)  would  be  research,  development,  production,  training,  oper¬ 
ations  (users),  support,  education,  planning  (within  each  Service  and  Joint),  costing,  decision  making  (pro¬ 
gram  managers.  Defense  Acquisition  Board  members  and  supporting  staff),  political  efforts  (supporters 
and  detractors),  and  decommissioning  and  disposal. 


distributed  simulation  could  contribute  within  the  DoD  and  its  suppliers  in  capitalizing  on 
these  advances  in  management  practices  and  technologies  to  improve  the  acquisition  pro¬ 
cess. 

Distributed  simulation  embodies  two  concepts.  First,  “simulation”  supporting 
acquisition  entails  the  ability  to  represent  the  properties  and  performance  of  physical  sys¬ 
tems  during  the  development  of  a  new  or  re-designed  product,  using  a  combination  of  live 
exercises  or  games,  constructive  models,  or  virtual  reality  [DSB  Sim  1993].  Simulation 
offers  two  capabilities.  The  first  is  visualization  which  provides  a  powerful  tool  for  com¬ 
munication  both  within  and  across  functional  communities.  Visualization  may  be  static,  as 
in  a  mechanical  component  or  assembly  displayed  within  a  CAD  software  package.  Visu¬ 
alization  could  also  be  dynamic,  as  when  the  movement  of  mechanical  components  is  dis¬ 
played.  Through  the  use  of  these  simulation  activities,  the  personnel  within  the  many 
disciplines  or  functional  communities  will  be  able  to  be  “immersed”  into  the  operation, 
manufacture,  repair,  etc.,  of  the  system  within  the  particular  environment  in  which  it  would 
operate.  The  second  capability  is  performance  modeling  which  augments  traditional  ana¬ 
lytical  tools  in  creating  a  workable  design.  Such  modeling  could  focus  on  simulating  phys¬ 
ical  performance,  such  as  the  stress  or  heat  dispersion  of  mechanical  components. 
Modeling  could  also  represent  the  operational  characteristics  of  a  component  or  weapon, 
such  as  reliability  or  combat  effectiveness. 

The  concept  of  “distributed  simulation”  entails  two  additional  capabilities.  The 
third  capability  is  a  common  information  base  that  can  be  accessed  from  or  at  different  loca¬ 
tions,  providing  team  members  with  the  most  recent  data  on  current  design  parameters, 
environments,  and  scenarios.  Finally,  the  fourth  capability  is  user  interaction.  Users  could 
range  from  designers  and  analysts  marking  up  drawings  from  remote  locations  to  test  pilots 
participating  in  simulated  air-to-air  combat  These  capabilities  allow  simulation  to  bridge 
both  geographic  and  functional  boundaries.  Combining  all  four  of  these  capabilities  into  a 
distributed  simulation  creates  an  effective  technology  for  creating  the  integrated,  functional 
teams  needed  to  implement  the  latest  management  practices. 

At  present,  simulations  of  various  forms  are  used  throughout  the  many  acquisition- 
related  DoD  functional  communities,  each  of  which  is  developing  the  modeling  and  simu¬ 
lation  tools  needed  to  perform  its  own  development  or  analysis  tasks.  There  is  some  degree 
of  information  sharing  among  those  simulation  tools,  but  the  unification  of  those  capabili¬ 
ties  in  a  broad-band  information  network  remains  a  vision  that  will  probably  take  decades 
to  fully  realize.  In  this  paper  we  examine  the  role  of  simulation  given  current  capabilities 
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as  well  as  how  the  contribution  of  distributed  simulation  will  grow  as  increasingly  sophis¬ 
ticated  technologies  come  on  line. 

Three  main  topics  are  discussed  in  this  paper.  First,  to  provide  a  framework  for 
examining  the  roles  of  modeling  and  simulation,  we  discuss  examples  of  conunercial  appli¬ 
cations  of  the  various  new  management  approaches  (e.g.,  IPPD,  concurrent  engineering, 
enterprise  integration,  and  TQM),  and  describe  a  functionally  integrated  system  acquisition 
process  based  on  the  central  concepts  within  those  approaches.  We  describe  how  manage¬ 
ment  practices  assist  in  coordinating  activities  among  functional  communities  and  how  this 
has  resulted  in  faster  convergence  on  designs  using  fewer  total  engineering  hours.  Through 
those  descriptions  and  examples  we  provide  benchmarks  and  principles  related  to  a  func¬ 
tionally  integrated  system  acquisition  process.  Second,  we  describe  the  current  uses  of  dis¬ 
tributed  simulation  in  DoD  development  programs.  Three  levels  of  simulation  are 
considered  in  order  to  capture  the  current  state  of  the  art  and  to  show  how  the  contribution 
of  simulation  will  grow  as  technology  advances.  Third,  we  examine  the  implications  of  our 
findings  for  the  DoD’s  simulation  investment  strategy  and  discuss  an  agenda  for  applying 
distributed  simulation  in  the  acquisition  process.  A  concluding  section  summarizes  our 
findings  and  observations,  and  outlines  a  series  of  steps  the  DoD  could  take  to  create  the 
simulation  environment  needed  to  support  a  functionally  integrated  system  acquisition  pro¬ 
cess. 
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11.  BENCHMARKS  AND  PRINCIPLES  FOR  AN  INTEGRATED 

DEVELOPMENT  PROCESS 


A  major  acquisition  program  will  involve  hundreds  of  government  officials  along 
with  scores  of  firms,  thousands  of  workers,  and  millions  of  work  hours.  Such  programs  will 
also  include  numerous  teams  from  several  functional  communities  for  defining  such 
aspects  as  program  concepts,  and  developing  product  and  process  technologies,  testing, 
logistics  planning,  budgeting,  and  oversight.  The  leadership  and  coordination  of  these 
teams  in  applying  leading  technologies  present  management  challenges  unmatched  by  vir¬ 
tually  any  other  modem  enterprise.  For  example,  the  current  generation  of  aircraft  in  devel¬ 
opment  incorporates  such  technologies  as  stealth,  high  performance  engines,  and  advanced 
avionics  and  control  systems.  Those  developments  require  the  application  of  new  materials 
and  manufacturing  processes,  along  with  highly  reliable  software  and  control  networks. 
The  multiple  technical  challenges  embodied  in  such  programs  present  complex  interactions 
among  teams  as  development  progresses. 

A,  LEARNING  AND  CONVERGENCE 

A  development  project  of  such  complexity  involves  learning  about  many 
unknowns  within  a  systematic  framework,  with  the  purpose  of  converging  on  a  design 
with  well-understood  properties.  Key  unknowns  might  include  speed,  weight,  power,  reli¬ 
ability,  manufacturing  process  requirements,  or  any  number  of  other  characteristics.  This 
learning  and  convergence  process  involves  numerous  cycles  of  analysis,  design,  fabrica¬ 
tion,  and  experimentation  both  within  each  of  the  functional  communities  and  across  such 
communities. 

Given  the  DoD’s  organizational  structure  and  the  legal  environment  in  which  the 
government  procures  weapon  systems,  we  have  identified  six  main  functional  conununi- 
ties  involved  in  this  learning  and  convergence  process: 

•  Requirements  and  concept  development:  Military  operators  and  technologists 
create  the  concepts  for  new  weapon  systems  based  on  military  mission  needs 
and  technological  opportunities.  Weapons  requirements  generally  evolve  from 


extensive  concept  studies  that  consider  tradeoffs  between  oi>erational  utility, 
costs,  support  requirements,  and  affordable  force  size. 

•  Engineering  design  and  manufacturing:  Teams  of  designers  and  engineers  cre¬ 
ate  detailed  designs  for  the  weapon  and  its  production.  Such  designs  evolve 
from  a  series  of  design  and  trade-off  analyses,  which  in  turn  evolve  into  a  pro¬ 
ducible  design  implementing  the  weapon  concept. 

•  Test  and  evaluation:  Independent  developmental  and  operational  testers  are 
required  under  law  to  validate  the  physical  performance  and  operational  utility 
of  new  weapon  systems.  Components  and  entire  systems  are  submitted  for  such 
testing  in  laboratories  and/or  on  test  ranges  throughout  the  development  pro¬ 
cess. 

•  Logistics  and  support:  As  a  program  develops,  the  logistics  and  related  commu¬ 
nities  plan  for  its  fielding  and  support.  Specifications  are  defined  for  training, 
maintenance,  and  spare  pans  purchases. 

•  Force  planning  and  budgeting:  Funding  of  programs  flows  through  the  DoD 
resource  allocation  process.  Resource  planners  (within  the  Services,  OSD,  other 
Executive  Office  organizations,  and  Congress)  are  responsible  for  reconciling 
demands  for  funding  so  that  forces,  modernization  plans,  and  budgets  are  appro¬ 
priately  balanced. 

•  Program  management:  Program  managers  are  faced  with  the  task  of  coordinat¬ 
ing  the  activities  of  all  these  functional  communities  whose  timely  support  is 
necessary  for  keeping  a  program  on  schedule,  controlling  costs,  and  attaining 
the  performance  objectives. 

The  interactions  among  functional  communities  can  be  visualized  as  an  iterative 
process  in  which  risks  and  uncertainty  are  systematically  reduced  through  analysis,  design, 
fabrication,  and  experimentation.  Over  time,  these  teams  consider  options  relating  to  such 
items  as  technology,  program  requirements,  and  costs — ultimately  they  converge  on  a 
design  for  a  fully  developed  system  (Figure  D-l). 

The  overall  goal  for  acquisition  program  managers  is  to  structure  the  development 
program  so  as  to  achieve  convergence  as  quickly  and  efficiently  as  possible.  These  man¬ 
agers  must  address  such  issues  as: 

•  How  many  cycles  of  exploration  are  needed  to  converge  to  a  program  design? 
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•  How  much  time  and  money  is  required  per  cycle? 

•  How  are  cycles  within  and  across  functional  communities  to  be  managed? 

•  Is  the  rate  of  convergence  balanced  across  functions? 

•  What  criteria  are  used  to  decide  when  convergence  is  reached? 


Figure  I1<1.  Convergence  in  the  Development  Process 


B.  NEW  MANAGEMENT  PRACTICES  IN  COMMERCIAL  FIRMS 

In  recent  years,  commercial  firms  have  developed  new  approaches  for  the  learning 
and  convergence  process  that  have  significantly  reduced  the  length  and  costs  of  develop¬ 
ment  programs.  Their  approaches  have  utilized  key  concepts  that  have  been  variously 
called  concurrent  engineering,  TQM,  enterprise  integration,  and  IPPD.  One  conunon  goal 
is  to  bring  the  functional  communities  together  from  the  very  outset  of  a  program  and  thus 
accelerate  the  learning  and  convergence  process.  An  integrated,  close-knit  team  defines  the 
critical  design  features  of  a  program  at  the  outset,  and  lays  the  groundwork  for  subsequent 
team  coordination.  In  commercial  firms,  this  entails  the  teaming  of  responsible  representa¬ 
tives  from  corporate  functional  communities,  including  management,  marketing,  account¬ 
ing,  and  finance,  as  well  as  designing  and  engineering  and  manufacturing. 


Four  examples  will  help  to  illustrate  how  these  new  management  practices,  key 
concepts,  and  advanced  technologies  have  been  applied.^  The  6rst  is  the  Ford  Taurus 
project.  Ford  formed  a  multi-functional  development  team  headed  by  Lew  Veraldi,  a  top 
Ford  engineer,  to  develop  the  Taurus.  Veraldi  set  a  development  goal  to  be  “best  in  class” 
for  each  of  about  400  product  characteristics,  and  met  about  three-fourths  of  these  goals. 
The  Taurus  development  project  took  seven  years  from  the  decision  to  develop  a  new  car, 
five  years  from  the  initial  sketches,  and  three  years  from  the  board  of  director’s  approval  of 
the  design  [Taub  1991,  p.  222].  This  was  significantly  less  than  typical  in  Detroit  at  that 
time. 

The  exhaustive  study  of  the  Japanese,  American,  and  European  auto  industries  per¬ 
formed  by  the  Massachusetts  Institute  of  Technology  (MIT)  concluded  that  automobile 
manufacturers  employing  such  new  advanced  management  practices,  such  as  used  in  the 
Taurus  project,  are  able  to  design  a  new  model  in  about  three-fourths  the  time — using  about 
two-thirds  the  engineering  hours — required  for  traditional  design  practices  [Womack 
1990].  Using  the  terminology  developed  by  the  MIT  team,  “lean  design”  entails  four  man¬ 
agement  principles  which  provide  a  good  description  of  concurrent  engineering  principles: 

•  Strong  project  leadership  that  controls  project  personnel,  budgets,  and  sched¬ 
ules. 

•  Project  teamwork  involving  a  multi-functional  core  team  of  personnel  working 
full  time  on  the  project. 

•  Design  team  communication  that  includes  clearly  defined  project  performance 
metrics  and  milestone  exit  criteria  which  are  needed  to  effectively  manage  risk. 

•  Simultaneous  development  that  includes  creation  of  production  equipment  in 
parallel  with  the  product. 

MIT  found  that  successful  development  programs  include  an  early  design  phase 
which  involves  all  of  the  functional  communities  in  defining  critical  design  parameters. 
These  decisions  form  the  guidance  for  detailed  development  work,  and  for  coordinating 
among  the  communities  as  the  development  project  advances. 

Lockheed  illustrated  a  similar  approach  in  a  space-based  system.  Lockheed  adopted 
a  “skunk  works”  management  approach  for  its  F-SAT  (Frugal  Satellite)  program,  headed 

*  Several  earlier  examples  can  be  found  in  Winner  [1988].  More  recent  examples  can  be  found  in  Nelson 
[1993]  and  Heim  [1993]. 


n-4 


by  a  strong  team  leader.  Lockheed  estimated  that  this  approach  reduced  development  time 
and  costs  by  50  percent  over  comparable  programs  developed  using  customary  methods. 
The  program  director,  Gary  Turner,  attributed  his  success  to  an  emphasis  on  direct  interac¬ 
tion,  and  ruthless  curtailment  of  oversight,  coordination,  and  review  [Turner  1992].  He 
emphasized  one  point  in  particular:  Lockheed  co-located,  in  a  single  building,  a  team  rep¬ 
resenting  each  of  the  functional  communities,  including  designers,  machinists,  and  cost  and 
financial  analysts.  At  the  outset,  procedures  were  established  to  maximize  flexibility  in 
designs  and  to  eliminate  bureaucracy.  Trial  and  error  and  changes  were  encouraged. 
Designers  could  directly  mark  up  blue  prints,  only  three  official  sets  of  prints  were  kept, 
and  one  person  was  assigned  responsibility  for  each  drawing. 

It  is  noteworthy  that  this  process  worked  well  without  sophisticated  information 
technology.  The  reason  was  that  team  members  were  able  to  communicate  directly  and 
work  with  up-to-date  paper  drawings.  The  Lockheed  experience  shows  that  effective  man¬ 
agement  of  the  design  team  and  its  process  is  the  main  requirement  for  improving  the 
development  process. 

A  third  example,  the  IBM  AS400  computer  development  program,  used  simulation 
extensively  to  augment  these  management  practices.^  IBM  established  cross-functional 
teams  that  integrated  software  and  hardware  design  and  manufacturing.  This  integrated 
approach  led  to  simplified  designs  and  also  reduced  the  part  count  from  10,000  for  the  ear¬ 
lier  S/38  to  4,000  for  the  AS400.  For  software  development,  IBM  broke  down  the  7  million 
lines  of  code  into  modules  which  were  spread  across  dedicated  software  development 
teams.  IBM  engineers  created  a  simulator  to  test  alternatives  until  they  converged  upon  a 
virtually  defect-firee  design.  IBM  was  able  to  develop  the  AS400  in  about  two  years — less 
than  half  the  time  required  for  its  predecessor  machine.  About  10  months  of  this  2-year 
design  cycle  reduction  was  due  to  IBM’s  use  of  simulation  of  hardware  design  options. 

Boeing  is  currently  extending  the  state  of  the  art  in  its  application  of  development 
technology  to  augment  these  new  management  practices  with  its  “paperless”  design  of  the 
new  B-777  aircraft  [Lucas  1993].  This  is  a  state-of-the-art  application  of  CAD,  CAE,  and 
CAM  which  is  costing  over  $1  billion  to  implement.  Boeing  creates  and  maintains  the 
design  in  a  computer  system  consisting  of  8  IBM  mainframes,  2,200  workstations,  and  a 
design  application  called  CATIA,  developed  by  Dassault  Systems  of  France.  CATIA  allows 
design  engineers  to  create,  configure,  specify,  and  check  their  parts  at  their  workstations. 


^  This  example  is  drawn  from  [Bowles  1991,  pp.  163-165]. 
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This  system  supports  238  functionally  integrated  “design/build”  teams  created  to  design 
the  aircraft.  These  teams  include  production  personnel,  supplier  representatives,  airline 
workers,  design  engineers,  and  many  others. 

The  B-777  will  be  the  first  commercial  jet  designed  entirely  on  a  computer  system, 
not  on  paper.  Boeing  will  build  it  without  creating  a  full  scale  production  mock-up.  This 
approach  follows  that  used  by  Northrop  in  the  B-2  production.  According  to  airplane  mak¬ 
ers,  the  single  biggest  problem  in  industrial  manufacturing  is  simply  parts  that  don't  fit 
together.  Manufacturers  have  traditionally  addressed  that  problem  by  constructing  a  com¬ 
plete  full-scale  mock-up  of  a  new  plane,  which  is  a  laborious,  time-consuming,  and  expen¬ 
sive  process,  just  to  find  out  if  parts  fit.  Parts-fit  checking  for  the  B-777  is  now  being  done 
via  CATIA.  Some  subsystems  have  been  mocked-up  where  CATIA  has  not  been  able  to 
mitigate  risks.  Manufacturing  experts  are  closely  watching  this  project  to  assess  the  results 
of  this  ambitious  experiment. 

These  examples  illustrate,  in  general,  how  today’s  commercial  leaders  are  structur¬ 
ing  their  development  programs  and  employing  new  management  practices  and  techniques 
to  speed  up  the  learning  and  convergence  process.  In  each  case  there  is  early,  intensive 
interaction  within  a  multi-functional  team,  during  which  major  design  parameters  are  ana¬ 
lyzed  and  key  decisions  are  made.  The  goal  is  to  make  change  easy  during  this  phase  in 
order  to  allow  as  many  iterations  as  possible  to  be  explored.  Hence  rapid  trial  and  error, 
feedback,  and  learning  are  emphasized  and  an  integrated  development  is  obtained.  Where 
technology  is  used,  it  is  to  support  this  process.  Indeed,  new  information  technologies  sup¬ 
port  such  integrated  development.  Distributed  design  and  engineering  simulation  tools 
could,  for  example,  create  a  virtual  co-location  for  design  teams  that  must  remain  geo¬ 
graphically  separated.  This  approach  has  allowed  such  integrated  development  to  be  used 
in  large  development  programs  such  as  the  B-777,  where  literal  co-location  of  the  design 
team  was  (and  remains)  impossible. 

C.  WHY  DO  THESE  NEW  MANAGEMENT  PRACTICES  SUCCEED? 

The  experiences  in  the  commercial  sector  provide  some  important  lessons  regard¬ 
ing  the  basic  management  principles  that  must  be  adopted  in  order  to  improve  the  develop¬ 
ment  process.  Large  corporations  contain  functional  communities  that  traditionally  have 
had  difficulty  working  together.  Designers,  manufacturers,  marketers,  accountants,  and  fin¬ 
anciers  see  programs  from  their  own  perspectives  and  have  difficulty  communicating 
across  functional  boundaries.  Moreover,  the  loyalty  of  the  design  team  members  to  their 


functional  communities  often  makes  it  impossible  for  program  managers  to  exert  necessary 
control  across  functional  lines.  The  traditional  process  of  sequential  or  “stovepipe”  devel¬ 
opment  causes  wasted  efforts,  rework,  and  delays.  The  goal  of  these  new  management  prac¬ 
tices  is  to  eliminate  such  waste  in  the  development  process  and  other  aspects  of  acquisition. 

To  help  reach  the  goal,  many  programs  are  now  structured  to  overcome  two  major 
shortcomings  that  have  plagued  large  development  programs  employing  a  stovepipe  devel¬ 
opment  approach.  The  first  shortcoming  has  been  a  lack  of  interaction  among  the  functional 
communities.  Traditionally,  this  has  led  to  sequential  development  stmctures,  in  which 
designs  are  passed  from  one  community  to  another  as  the  development  program  progresses. 
This  approach  is  both  time  consuming  and  inefficient.  Significant  time  is  lost,  for  example, 
when  an  engineering  design  is  “tossed  over  the  transom”  to  the  manufacturing  specialists 
who  must  then  learn  the  design  and  figure  out  how  to  manufacture  it.  Moreover  because  the 
designers  did  their  work  with  incomplete  feedback  from  the  production  or  support  commu¬ 
nities,  their  designs  may  later  need  to  be  modified  when  these  communities  get  their  chance 
to  review  the  designs.  Corrections  come  belatedly  as  the  inputs  from  other  teams  and  func¬ 
tional  communities  are  received.  An  important  goal  of  these  new  management  practices  is 
to  shorten  the  feedback  loop  which  in  turn  accelerates  the  development  process. 

The  second  shortcoming  with  the  stovepipe  development  approach  is  a  failure  to 
converge  on  a  successful  design  prior  to  initiating  full-scale  engineering  and  manufacturing 
development.  Given  the  cumbersome  feedback  process,  the  lack  of  clear  risk-management 
metrics,  and  the  need  to  keep  programs  moving,  programs  frequently  begin  engineering 
and  manufacturing  development  before  all  the  technical  problems  have  been  recognized 
and  solved.  At  this  stage,  the  cost  of  rectifying  such  problems  multiplies.  Fixes  involve 
reworking  earlier  designs,  and  the  coordination  cost  of  these  fixes  is  magnified  by  the  fact 
that  there  are  large  numbers  of  people  doing  detailed  design  work.  Because  the  number  of 
workers  involved  in  a  program  increases  by  several  multiples  when  a  program  enters  engi¬ 
neering  and  manufacturing  development,  the  costs  skyrocket  for  each  day’s  delay  caused 
by  technical  difficulties. 

Both  shortcomings — ineffective  coordination  and  incomplete  convergence — have 
caused  substantial  delays  and  cost  growth  in  defense  acquisition  programs.  In  early  pro¬ 
gram  phases,  programs  are  often  delayed  because  of  disagreements  in  defining  a  program 
concept  (Table  II- 1).  Subsequendy,  incomplete  convergence  gives  rise  to  technical  prob¬ 
lems.  Delays  in  the  engineering  and  manufacturing  development  phase  of  a  representative 


Table  II- 1.  Convergence  and  Risk  Management  in  DoD  Prc^rams^ 


Program 

Phase 

Average  Schedule  in  Months 

Plan  Added  Time 

Reasons  for  Program 
Stretch 

Demonstration 
&  Validation 

30  mos. 

25  mos. 

Lack  of  Convergence  in 
Program  Definition 
(8  of  1 1  cases) 

Full-Scale 
Development 
&  Low-Rate 
Initial  Produc¬ 
tion 

51  mos. 

33  mos. 

Technical  Difficulties 
(20  of  27  cases) 

Lack  of  Convergence  in 
Program  Definition 
(5  of  27  cases) 

a.  Source:  [Bicksler  1991], 


sample  of  recent  programs  added  60  percent  to  planned  development  times.  Required  bud¬ 
gets  were  also  increased:  at  least  one-third  of  the  programs  required  more  than  twice  as 
much  development  funding  as  initially  planned.^  New  management  practices  and  develop¬ 
ment  technologies  that  help  to  identify  and  eliminate  these  technological  problems  prior  to 
entering  engineering  and  manufacturing  development  could  significantly  reduce  program 
schedules  and  costs. 

NASA  found  a  similar  pattern  in  reviewing  the  history  of  its  major  programs.  When 
initial  design  phase  investments  were  correlated  with  subsequent  cost  and  schedule  perfor¬ 
mance,  they  found  that  programs  experienced  far  less  cost  and  schedule  growth  when  a 
greater  share  of  program  resources  was  devoted  to  the  initial  design  process.^  NASA  thus 
concluded  that  significant  program  cost  savings — roughly  25  percent  of  program  develop¬ 
ment  costs — could  be  obtained  by  ensuring  that  adequate  design  convergence  is  achieved 
before  full-scale  development  begins. 

These  new  management  practices  (especially  concurrent  engineering)  entails  a  new 
structure  for  managing  development  programs.  Somewhat  more  effort  is  to  be  expended 
earlier  in  the  development  process,  when  a  multi-functional  team  defines  the  main  program 


•j 

These  schedule  slippages  and  funding  increases  represent  a  snapshot  of  a  sample  of  programs  within  the 
engineering  and  manufacturing  development  phase  at  the  time  the  study  was  conducted.  Many  of  these 
programs  did  experience  even  further  slippage  and  funding  growth  (beyond  that  reported  in  the  source  doc¬ 
ument)  before  completing  the  development  phase.  See  Bicksler  [1991]. 

^  NASA  found  that  programs  that  invested  S  percent  of  planned  program  funds  in  the  initial  design  phase 
(phase  A/B  in  NASA’s  terminology)  had  subsequent  overruns  averaged  30  percent  When  10  percent  was 
invested  in  this  phase,  program  overruns  were  only  5  percent  [NASA  1992]. 
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development  parameters  (Figure  II-2).  Most  of  the  design  effort  is  accomplished  during  the 


Figure  II-2.  Restructuring  the  Development  Process 

Source:  LaBauve  ( 1992,  p.  28] 


initial  design  phase.  Less  effort  is  required  for  design  modifications,  and  because  risk  is 
managed  better,  substantially  less  redesign  is  required  for  engineering  changes  during  full- 
scale  development.  Fewer  total  engineering  hours  are  required,  and  the  development  time 
is  substantially  reduced. 

As  experience  is  gained  with  these  new  management  practices,  the  reasons  for  their 
success  are  becoming  clear.  By  using  these  practices,  companies  have  been  able  to  acceler¬ 
ate  the  cycles  in  the  iterative  process  of  analysis  and  experimentation,  which  allows  the 
functional  conununities  to  converge  more  quickly  on  a  design.  The  practices  also  allow  rap¬ 
id  consideration  of  many  options  within  each  functional  communities,  and  facilitate  com¬ 
munication  across  functional  communities.  A  corollary  benefit  is  the  improved 
management  of  risk  within  this  process.  With  enhanced  communication  among  partici¬ 
pants,  problems  or  risks  surface  much  earlier  and  can  be  resolved  before  major  additional 
expenses  are  incurred. 

D.  THE  ROLE  OF  DISTRmUTED  SIMULATION 

Leading  commercial  firms  are  exploiting  simulation  capabilities  to  augment  these 
new  practices  to  further  streamline  and  accelerate  the  design  convergence  process  in  devel- 
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oping  new  products.  Within  appropriate  management  structures,  commercial  companies 
have  used  simulation  to  contribute  to: 

•  Faster  cycles  of  analysis  and  experimentation  within  each  functional  commu¬ 
nity. 

•  Faster  and  clearer  communication  across  functional  communities. 

•  Clearer  understanding’  of  and  consensus  on  major  design  parameters. 

•  Improved  understanding  and  management  of  risks. 

For  many  readers  of  this  paper — those  with  the  benefit  of  hindsight  and  a  perspective  from 
outside  of  the  organizations  involved  — these  management  practices  may  seem  obvious  and 
straightforward  to  apply.  However,  the  changes  needed  in  their  adoption  are  very  difficult 
to  make  because  they  threaten  existing  organizational  cultures  and  hierarchies.  The  DoD 
acquisition  community  faces  even  greater  barriers  than  faced  by  commercial  firms:  tradi¬ 
tional  cultural  and  organizational  barriers  are  at  least  as  daunting  within  the  government  as 
in  the  commercial  world;  there  are  significant  legislative  and  regulatory  barriers  imposing 
arms-length  checks  and  balances  within  the  government  that  make  it  difficult  for  commu¬ 
nities  to  work  together;  and  the  DoD  lacks  the  profit-loss  incentive  of  the  competitive  mar¬ 
ketplace  which  has  strongly  motivated  commercial  organizational  and  management 
innovation.  We  will  return  to  these  issues  in  the  final  section  when  we  review  the  experi¬ 
ence  of  some  commercial  firms  in  implementing  these  new  practices,  and  consider  some  of 
the  issues  that  the  DoD  will  have  to  address  in  adapting  those  practices  and  distributed  sim¬ 
ulation  to  defense  development  acquisition. 


III.  DISTRIBUTED  SIMULATION  IN  DOD’S  DEVELOPMENT 

PROGRAMS 


The  degree  to  which  distributed  simulation  can  support  acquisition  programs  within 
DoD — both  today  and  in  the  future — depends  on  the  power  and  acceptance  of  the  available 
distributed  simulation  tools.  These  can  be  measured  in  terms  of  the  degree  to  which  avail¬ 
able  tools  provide  the  four  capabilities  outlined  in  the  introduction:  visualization,  perfor¬ 
mance  modeling,  common  information  bass,  and  user  interaction.  To  gain  an  understanding 
of  available  simulation  technologies,  this  section  examines  the  current  state  of  the  art  and 
the  potential  future  application  of  distributed  simulation  in  defense  development  programs. 

In  this  paper  we  consider  three  general  levels  of  distributed  simulation,  which  rep¬ 
resent  the  increasingly  powerful  tools  that  will  be  available  to  support  the  DoD  acquisition 
processes.  These  simulation  technology  advances  are: 

•  Functional  distributed  simulation:  Provides  visualization  and  performance 
modeling  within  a  functional  community.  Most  current  simulations  are  of  this 
type. 

•  Cross-Junctional  distributed  simulation:  Provides  common  data  and  user  inter¬ 
action  across  some  functional  conununity  boundaries  and  permits  some 
enhanced  functional  capabilities.  Limited  cross-functional  capabilities  also 
exist  today. 

•  Functionally  integrated  simulation:  Represents  the  long-term  vision  in  which 
distributed  simulation  would  eliminate  the  language  barriers  across  and  within 
all  of  the  functional  communities.  This  would  permit  engineering  designs  to  be 
reviewed  and  evaluated  by  the  communities  by  using  their  customary  analysis 
tools  and  utilizing  an  enterprise  information  network  connect  with  conunon 
information  bases. 

We  shall  begin  by  describing  the  current  status  of  distributed  simulation  within  each 
of  the  six  functional  communities,  and  then  review  simulation’s  role  in  improving  cross¬ 
functional  communication  and  coordination.  Examples  based  on  site  visits,  interviews,  and 


available  reports  illustrate  how  distributed  simulation  has  been  used.  These  examples  do 
not  represent  an  exhaustive  review,  but  we  believe  they  provide  a  representative  view  of  the 
lands  of  simulation  capabilities  that  are  in  use. 

A.  SIMULATION  WITHIN  FUNCTIONAL  COMMUNITIES 

Simulations  within  individual  functional  communities  include  those  that  offer  visu¬ 
alization  and  performance  modeling  capabilities  for  a  single  functional  conununity,  without 
necessarily  providing  information  which  automatically/electronically  passes  through  net¬ 
works  which  crosses  functional  lines.  For  example,  wind-tunnel  simulations  assist  engi¬ 
neering  designers  in  creating  new  wing  designs.  Combat  simulations  allow  concept 
developers  to  examine  effectiveness  in  a  combat  environment.  Such  tools  permit  each  com¬ 
munity  to  do  its  work  faster,  thus  contributing  to  an  acceleration  of  the  learning  and  con¬ 
vergence  process.  These  advances  have  been  spurred  by  the  continuously  expanding 
capabilities  within  computer  hardware  and  software  disciplines. 

These  kinds  of  simulation  tools  already  are  common  within  each  of  the  DoD’s  func¬ 
tional  communities.  Since  each  community  has  a  unique  focus,  the  tools  they  have  devel¬ 
oped  tend  to  be  specialized.  Such  simulation  tools  generally  fall  into  one  of  four  families: 

•  The  combat  performance  simulations  used  mostly  by  concept  developers,  train¬ 
ers,  testers,  and  force  planners. 

•  The  physical  simulation  exemplified  by  performance  models,  engineering  sim¬ 
ulations,  and  process  models  which  are  used  mostly  by  designers,  engineers, 
and  manufacturers. 

•  Availability  and  support  simulation  models  used  mostly  by  operational  and 
logistics  planners. 

•  Cost  and  schedule  models  and  simulations  used  mostly  in  budgeting  as  well  as 
in  structuring  and  managing  programs. 

The  nature  and  role  of  simulation  will  therefore  differ  across  functional  communities  and 
families  of  simulation  (Table  III-l).  The  appropriate  role  of  the  human  in  the  simulation 
varies  depending  upon  the  application  because  not  all  functions  require  manned  simula¬ 
tions  to  address  their  concerns.  But  in  some  areas,  particularly  requirements  and  concept 
development  and  test  and  evaluation,  the  ability  to  conduct  manned  simulations  is  essential. 
The  role  of  simulation  in  each  of  the  functional  communities  is  discussed  in  turn. 


Table  ni-l.  Examples  of  Simulations  Used  Within  Functional  Conununities 


Functional 

Community 

Simulation  Tools 
(Simulation  Family^) 

Simulation 

Examples 

Program 

Examples 

Requirements  and 
Concept  Develop¬ 
ment 

System,  unit,  and 
force-level  effective¬ 
ness  (combat) 

SIMNET 

MlAl 

Military  worth  (com¬ 
bat) 

Domed  Flight  Simu¬ 
lators 

AMRAAM,  F-15,  F/ 
A-18,F-22 

Engineering  Design 
and  Manufacturing 

Physical  performance 
(physical) 

IBM 

AS4(X) 

Human  factors  (phys¬ 
ical) 

CATIA 

B-777 

Manufacturability 

(physical) 

Northrop 

B-2 

Test  and  Evaluation 

Hiysical  performance 
(physical) 

SIMNET 

FAADS,  ADATS 
NLOS/FOG-M 

Operational  effec¬ 
tiveness  (combat) 

Domed  Flight  Simu¬ 
lators 

F-22,F/A-18,F-15 

Logistics  and  Support 

Reliability  (physical) 

RAMCAD 

B-52,  M109 

Support  require¬ 
ments  (availability 
and  support) 

Logistics  Capability 
Model  (LCAM) 

F-22 

Force  and  Budget 
Planning 

System,  unit  and 
force-level  effective¬ 
ness  (combat) 

TACWAR 

JANUS 

FOFA/ATACMS 
FOG-M,  NLOS, 

MlAl 

Resource  planning 
(cost  and  schedule) 

FACS 

Numerous  Programs 

Program  Manage¬ 
ment 

Program  structure 

Cost  estimation 

IDEF,  FORESIGHT 

MUAVATOL-UAV 

Program  manage¬ 
ment  (cost  and  sched¬ 
ule) 

Parametric  cost  mod¬ 
els 

a.  The  simulation  families  are  defined  in  the  text,  Section  HI. A. 
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1.  Requirements  and  Concept  Development 

Simulations  have  been  and  are  being  used  extensively  in  defining  concepts  in  some 
warfare  areas.  They  support  the  demonstration  of  new  technologies,  concepts,  and  tactics, 
and  they  support  the  analysis  of  man-in-the-loop  design  factors. 

The  premier  applications  are  in  air-to-air  combat.  One  noteworthy  example  is  the 
operational  utility  evaluation  conducted  for  the  AMRAAM  missile  in  the  early  1980s.'  At 
that  time,  AMRAAM  was  a  conceptual  design  for  a  long-range,  fire-and-forget  air-to-air 
missile.  There  was  some  question  as  to  whether  such  a  missile  would  be  operationally 
effective,  given  the  problems  in  identifying  friend  vs.  foe  aircraft  at  beyond  visual  range. 
To  address  these  concerns,  an  extensive  simulated  operational  test  of  the  AMRAAM  was 
conducted  using  domed  flight  simulators  at  McDonnell  Douglas.  Simulations  were  con¬ 
ducted  because  the  cost  of  live  tests  was  prohibitive.  The  simulations  included  red  and  blue 
teams  of  aircraft.  Pilot’s  tactics  evolved  rapidly  as  they  gained  experience  in  using  and 
eluding  the  missile,  illustrating  the  importance  of  manned  simulation  in  concept  develop¬ 
ment.  The  tests  concluded  that  the  missile  was  useful  but  mostly  because  of  its  fire-and- 
forget  features  rather  than  its  long-flight  range.  The  tests  also  uncovered  problems  with  the 
AMRAAM  seeker  that  eventually  were  worked  out  in  engineering  development. 

Air-to-air  simulators  have  also  been  used  extensively  for  the  F- 15  and  F/A-18  pro¬ 
grams  at  McDonnell  Douglas,  and  they  play  a  significant  role  in  the  ongoing  F-22  program 
as  well.  Two  factors  have  contributed  to  the  success  of  simulation  in  the  air-to-air  warfare 
area.  The  first  is  that  simulation  technology  is  weU  developed  and  accepted  because  of  its 
long-standing  use  in  the  training  community.  The  second  is  that  such  applications  involve 
a  rather  simple  environment  to  model,  and  can  be  meaningfully  undertaken  with  relatively 
few  simulated  weapons. 

Much  attention  is  given  today  to  attempts  to  extend  manned  distributed  simulation 
to  the  ground  combat  environment.  One  example  is  presented  by  the  defense  Simulator 
Network  (SIMNET)  studies  in  helping  to  establish  the  requirements  for  the  M1A2  tank 
[USA  1988].  A  recent  Defense  Science  Board  study  [DSB  Sim  1993]  estimated  that  such 
simulations  provided  order-of-magnitude  savings  in  concept  development  over  traditional 
development  methods.  In  the  mid-1980s,  a  live  test  of  possible  upgrades  of  the  Ml  loader 

*  Private  communications  with  Mr.  John  Shea,  Institute  for  Defense  Analyses,  Alexandria,  Wginia,  and  Mr. 

Harry  Passmore,  McDonnell  Aircraft  Company,  St.  Louis,  Missouri. 

^  This  example  is  based  on  the  DSB  study  cited.  Costs  are  the  marginal  costs  of  conducting  the  tests  and 
simulations,  and  do  not  include  the  costs  of  developing  the  simulators. 


II1-4 


and  fire-control  systems  took  two  years  and  cost  $40  million.  Despite  those  costs,  only  one 
design  alternative  was  able  to  be  tested.  Subsequently,  a  1986  test  using  modified  aircraft 
dome  simulators  allowed  a  single  version  of  an  Ml  A2  configuration  to  be  simulated  in  six 
months  at  a  cost  of  $1  million.  By  1992.  with  the  availability  of  SIMNET,  it  was  possible 
to  examine  four  variations  of  the  M 1 A2  design  in  only  three  months  at  a  cost  of  $600,000. 
The  SIMNET  simulations  played  a  substantial  role  in  defining  the  concept  for  that  program. 

Other  examples,  such  as  the  use  of  the  Theater  Air  Command  and  Control  Simula¬ 
tion  Facility  (TACCSF)  for  early  proof  of  principle  tests  for  friend  vs.  foe  identification, 
suggest  that  these  simulation  methods  may  be  applicable  for  command  and  control  systems 
as  well.  These  examples,  while  selective,  show  the  extensive  use  already  being  made  of 
simulation  in  setting  and  defining  program  concepts  and  requirements. 

A  key  problem  in  the  establishment  of  the  requirements  has  been  the  mutual  accep¬ 
tance  of  the  measures  of  performances  that  are  to  be  used  throughout  the  life  of  the  pro¬ 
gram.  Those  measures  should  be  established  and  used  in  each  of  the  functional 
communities  to  conduct  such  efforts  as  formulating  the  design,  conducting  the  cost  and 
operational  effectiveness  analyses  (COEAs),  and  evaluating  the  design  during  tests  and 
evaluations 

2.  Physical 

Performance,  engineering,  and  process  models  and  simulations  are  extensively 
used  within  the  engineering  and  manufacturing  communities.  Commercial  software  devel¬ 
opers  are  creating  a  wide  range  of  engineering  simulation  tools  that  allow  engineers  to  sim¬ 
ulate  different  concepts  and  designs.^  They  allow  the  physical  characteristics  of  designs  to 
be  modeled  and  simulated.  Stress  analyses,  heat  transfer,  electromagnetic  properties,  and 
dynamic  properties  of  components  can  now  be  examined.  These  software  packages  can 
directly  read  by  standard  CAD/CAE  data  bases.^  In  many  cases,  manufacturing  process 
engineers  can  tap  into  these  data  bases  and  examine  manufacturing  process  issues  using 

^  The  design  and  engineering  software  packages  cuitently  available  include  modules  for  high-level  concept 
development  and  detailed  engine^^rir^g  design.  Many  include  internal  analyses  tools  such  as  stress  analysis. 
Many  provide,  as  options,  modules  that  permit  manufacturing  process  design  issues  to  be  addressed.  The 
design  data  bases  can  also  be  exported  for  use  in  engineering  simulation  models. 

Some  of  the  brand  names  in  design  and  engineering  tools  that  are  currently  on  the  market  include  the  fol¬ 
lowing:  Intrgraph’s  MicroStation,  CADKey’s  CADKEY,  Autodesk’s  AutoCAD,  Computervision’s  Per¬ 
sonal  Designer,  and  Adra’s  CADRA-III.  A  n^idly  growing  competitCH'  is  Parametric  Technology’s  family 
of  tools  including  PRO/Design,  PRO/Engineer,  and  a  range  of  manufacturing  application  tools.  Other 
major  names  include  IBM’s  CADAM,  the  Manufacturing  and  Consulting  Service’s  ANVIL-5000,  and 
Dassault’s  CATIA. 
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their  own  simulations.  Applications  of  these  technologies  by  IBM  and  Boeing  were 
described  in  the  second  section.  Current  technology  thus  permits  extensive  interaction 
among  concept  designers  and  product  and  process  engineers,  supporting  the  implementa¬ 
tion  of  the  principles  of  these  new  management  practices. 

The  use  of  these  engineering  tools  is  also  commonplace  among  defense  firms.  One 
well-known  application  illustrating  the  state  of  the  art  is  Northrop ’s  extensive  simulation 
of  the  B2  structural  designs.  Northrop  credits  its  simulation  activities  with  allowing  it  to 
skip  the  usual  full  scale  production  mock-up  and  still  fabricate  complex  components  with 
no  scrap  or  rework  [Winner  1993]. 

Engineering  design  and  manufacturing  represents  the  most  mature  application  of 
simulation  tools  of  all  DoD’s  functional  communities.  It  also  is  an  important  example  of 
dual  use  technology.  Advances  have  been  driven  by  commercial  developers  who  continue 
to  improve  the  available  tools  and  data.  The  DoD  needs  to  keep  abreast  of  commercial 
developments  and  adapt  them  as  needed  for  application  in  defense  programs. 

3.  Test  and  Evaluation 

Simulation  has  long  been  used  within  the  test  and  evaluation  community  to  comple¬ 
ment  traditional  tests.  Engineering  simulations  of  physical  characteristics  support  develop¬ 
mental  testing,  which  is  performed  in  the  design  and  development  phase  of  system 
acquisition.  They  evaluate  the  designs  by  addressing  the  total  system,  subsystems,  parts, 
and  components.  They  also  seek  to  demonstrate  actual  performance.  Combat  simulations 
help  guide  and  augment  operational  tests,  and  help  incorporate  human  factors.  Operational 
testing  is  conducted  to  determine  the  operational  effectiveness  and  suitability  of  a  system 
under  realistic  combat  conditions.  Because  of  the  very  nature  of  operational  testing,  virtual 
or  constructive  simulations  cannot  fully  replace  live  testing.  Nevertheless,  simulation  has 
an  important  role.  The  application  of  distributed  simulation  in  each  of  these  phases  of  test¬ 
ing  is  discussed  below  for  both  developmental  and  operational  testing. 

Engineering  simulations  are  commonly  used  in  developmental  testing  and  evalua¬ 
tion.  Some  examples  include  wind  tunnel  models;  mobility  models;  reliability,  survivabil¬ 
ity,  and  maintainability  models;  and  human  factors  and  control  models.  Various  component 

^  A  leading  supplier  of  such  modeling  tools  is  the  MacNeal-Schwendler-Aries  team,  which  offers  the  Con- 
ceptStadon  tfesign  system  and  modeling  tools  including  MSC/NASTRAN,  which  performs  structural  and 
heat  transfer  analysis;  NSC/EMAS,  which  performs  electromagnetic  analysis;  and  MSC/DYTRAN,  which 
conducts  high-speed  fluid-structure  interacdon  analysis.  This  later  analysis  is  used,  fen*  example,  to  model 
the  effect  of  bird  strikes  on  aircraft  engines  and  canopies. 


simulations  (such  as  missile  fly-out  models)  are  used  in  system  testing  to  reduce  costs.  Used 
in  conjunction  with  developmental  testing,  these  tools  can  significantly  reduce  the  time  and 
cost  required  to  demonstrate  the  physical  performance  of  new  systems  and  their  compo¬ 
nents.^ 

The  role  of  simulation  changes  somewhat  as  testing  transitions  from  developmental 
to  operational  issues.  In  operational  testing,  simulations  based  on  developmental  test 
results  are  used  to  design  and  evaluate  tests,  and  to  overcome  the  limitations  and  constraints 
of  a  field  test  environment.  Several  examples  illustrate  the  applicability  of  simulation  in 
operational  testing.  The  manned  simulators  at  McDonnell  Douglas’s  domed  flight  simula¬ 
tion  facility  were  extensively  used  in  developing  better  operational  testing  scenarios  in 
developing  the  F- 15  aircraft.  Simulation  of  F-15  flight  tests  helped  considerably  in  design¬ 
ing  specific  details  of  actual  flight  tests,  as  well  as  in  constructing  the  overall  matrix  of  flight 
tests  needed  to  address  design  uncertainties. 

An  appreciation  of  the  role  that  simulation  can  play  in  improving  cost  effectiveness 
in  testing  is  provided  by  comparing  the  resources  expended  during  the  planning  and  then 
the  execution  of  the  developmental  testing  for  the  Forward  Area  Air  Defense  System 
(FAADS).  Constructive  simulations  using  JANUS  helped  to  establish  the  key  performance 
measures,  then  distributed  simulations  using  SIMNET  helped  to  plan  the  actual  tests,  and 
then  less  expensive  (than  initially  planned)  field  tests  were  conducted.  Considerable 
resources  can  be  saved  with  effective  planning  using  models  or  simulations  to  complement 
field  test  time^  (Table  III-2). 


Table  111-2.  Resources  Expended  in  Planning  and  Execution  of  FAADS  Field  Tests 


Phase 

JANUS 

Man-Days 

SIMNET 

Man-Days 

Field  Test 
Man-Days 

Preparation 

21 

46 

2,345 

Exercise 

105 

550 

5,067® 

Analysis 

74 

25 

600 

Total 

200 

621 

8,012 

a.  Includes  extensive  reliability  and  specified  stand-alone  tests 


^  Ford  Motor  Co.’s  Howard  Crabb,  manager  of  computer-aided  engineering  (CAE)  for  Ford’s  Alpha  design 
division,  estimates  that  use  of  the  “ConceptStadon  solid  modeling  CAE  tool  combined  with  MSC/NAS- 
TRAN  has  reduced  the  cycle  time  for  designing  components  by  40%  to  70%.”  (See  [MacNeal  1992].) 


Another  important  application  is  in  the  ongoing  development  of  test  plans  for  the 
F-22.  The  Air  Force  presently  is  conducting  an  18-month  study  to  determine  the  appropri¬ 
ate  set  of  tests  for  comparing  the  effectiveness  of  the  F-22  against  the  F- 15  [USAF  1992]. 
These  tests  will  occur  as  part  of  the  F-22  OT&E.  This  study  is  relying  on  a  combination  of 
the  TAC  BRAWLER  air  combat  model  and  exercises  using  Lockheed's  domed  simulator 
facility.  Hundreds  of  model  runs  will  be  undertaken  to  identify  the  parameters  that  best  dis¬ 
criminate  the  performance  of  the  comparison  aircraft.  Domed  simulations  will  then  focus 
on  the  effect  of  varying  assumptions  about  this  set  of  variables  and  determine  measures  that 
best  define  “mission  success.”  The  domed  simulators  will  be  calibrated  when  data  become 
available  from  the  early  flight  tests,  so  that  the  simulators  can  be  used  to  extrapolate  to  envi¬ 
ronments  that  are  not  feasible  in  the  live  tests  (e.g.,  higher  density  threat  environments). 

As  we  have  shown,  simulation  plays  important  roles  in  both  developmental  and 
operational  test,  and  its  contribution  will  increase  as  simulation  technology  advances. 
However,  simulation  can  never  be  expected  to  frilly  replace  the  need  for  real  tests.  Simula¬ 
tion — like  other  performance  modeling  tools — is  limited  by  the  range  of  experience 
embodied  in  its  analytical  structure  and  data  bases.  There  are  valid  concerns  about  such 
data  issues  as  being  able  to  get  wide  acceptance  of  data  from  manned  simulations,  being 
able  to  validate  results  of  simulations  versus  data  from  test  ranges,  and  being  able  to  extrap¬ 
olate  with  some  degree  of  assurance  from  the  simulated  test  environment  to  other  environ¬ 
ments.  Simulation  will  be  useful  for  extrapolating  to  new  designs,  environments,  and 
scenarios  in  some  cases,  but  in  other  cases  extrapolation  errors  may  invalidate  the  results. 
This  question  of  extrapolation  error  is  at  the  heart  of  the  W&A  of  simulations,  and  it  is 
also  the  key  limiting  factor  in  its  use  for  developmental  and  operational  testing.  Although 
simulations  are  designed  to  capture  known  physical  principles,  DoD  development  pro¬ 
grams  are  often  designed  to  push  the  envelope  of  well-known  technology.  Hence,  there  will 
nearly  always  be  some  uncertainty  and  risk  of  extrapolation  error  when  new  development 
programs  are  simulated.  Nevertheless,  as  the  preceding  examples  show,  simulation  used  in 
conjunction  with  testing  is  a  powerful  development  tool. 


^  In  a  similar  example,  SIMNET  was  used  in  evaluating  ADATS  test  range  results  to  show  that  simulation 
was  able  to  predict  the  major  measures  of  performance.  This  test  suggests  that  SIMNET  could  usefully  be 
applied  in  designing  subsequent  tests.  If  a  simulation  can  lead  to  the  same  general  conclusion  as  a  test  and 
i^ntify  the  factors  driving  the  results,  then  simulation  can  provide  an  important  tool  for  early  operational 
assessments.  Similarly,  simulation  may  be  useful  in  augmenting  test  data  that  was  limited  by  s^ety  con¬ 
siderations,  expense,  availability  of  test  ranges,  or  weather.  All  in  all,  the  use  of  simulation  can  complement 
testing,  making  testing  more  effective  and  economizing  to  some  degree  on  the  expense  of  field  tests. 
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4.  Logistics  and  Support 

The  capabilities  that  are  being  developed  to  strengthen  analysis  of  logistics  and  sup¬ 
port  issues  fall  into  two  main  categories.  The  first  are  those  that  are  bringing  reliability  and 
maintainability  concerns  within  the  engineering  environment.  Among  these  is  the  Contin¬ 
uous  Acquisition  and  Life-cycle  Support  program  (CALS),  formerly  called  the  Computer- 
Aided  Acquisition  and  Logistics  Support  program.  Although  CALS  is  not  itself  a  simula¬ 
tion  tool,  its  goal  is  to  create  the  data  bases  that  would  also  be  used  in  simulating  support 
issues  as  they  relate  to  the  design  process.  It  currently  includes  three  main  activities: 

•  Accelerating  the  introduction  of  reliability,  maintainability,  and  supportability 
considerations  into  the  computer-aided  design  process. 

«  Enhancing  the  capabilities  within  defense  industry  to  supply  technical  data  to 
the  government  in  digital  form. 

•  Enhancing  the  capabilities  within  the  government  to  properly  receive  and  use 
these  data. 

CALS  data  exchange  standards  are  now  important  vehicles  for  the  management  and 
distribution  of  maintenance  “tech  data.” 

An  initial  thrust  of  CALS  is  RAMCAD  (Reliability  And  Maintainability  in  Com¬ 
puter-Aided  Design).  It  focuses  on  the  use  of  computer-aided  design  technology  to  contin¬ 
ually  assess  and  improve  the  reliability,  maintainability,  and  supportability  characteristics 
of  a  product  throughout  the  design  phase.  The  current  focus  of  RAMCAD  is  on: 

•  Improving  methodologies  for  incorporating  support  issues  in  design  analysis. 

•  Improving  methodologies  to  conduct  tradeoffs  among  support  measures. 

•  Improving  methods  to  allow  tradeoffs  among  the  different  engineering  disci¬ 
plines. 

The  second  kind  of  logistics  simulation  capability  employs  reliability  and  maintain¬ 
ability  statistics  to  estimate  support  requirements.  For  example,  the  Air  Force  is  using  the 
Logistics  Capability  Assessment  Model  (LCAM)  to  estimate  the  logistics  support  require¬ 
ments  for  the  F-22  aircraft.  This  model  employs  Monte  Carlo  techniques  to  simulate  com¬ 
ponent  failure  rates  in  estimating  maintenance  manpower  and  spare  parts  requirements. 

It  is  apparent  firom  these  examples  that  extensive  efforts  are  underway  to  gain  not 
only  better  understanding  of  the  logistics  capabilities  and  deficiencies  but  also  to  incorpo- 


rate  them  into  weapon  support  considerations  within  the  design  process.  This  is  one  area 
where  the  acquisition  community  has  clear  responsibility  for  investments  in  technology, 
and  this  should  remain  a  focus  for  future  efforts. 


5.  Force  Planning  and  Budgeting 

The  DoD  has  relied  extensively  on  models  and  simulations  in  its  systems  analysis 
approach  for  evaluating  force  structures  and  for  planning  force  modernization.  Many  the¬ 
ater-level  models  and  simulation  tools  are  in  use  today  to  assist  in  designing  forces  and 
planning  for  modernization.  These  plans  underlie  the  development  of  DoD  budgets  even 
when  they  are  not  used  directly  in  the  budgeting  process.  One  good  example  is  the  model¬ 
ing  done  in  the  1980s  in  creating  the  deep  strike,  follow-on-forces  attack  (FOFA)  modern¬ 
ization  strategy  within  the  Office  of  the  Secretary  of  Defense  (OSD)  and  the  Army. 
Extensive  analyses  considered  force  structure  options  for  deep  attack  weapons  and  their 
appropriate  balance  with  direct  fire  weapons.  This  kind  of  analysis  is  commonly  conducted 
to  establish  force  structure  requirements  for  new  systems. 

The  Advanced  Research  Project  Agency’s  (ARPA)  recent  WAR  BREAKER  dem¬ 
onstration  is  a  good  illustration  of  how  distributed  simulation  could  be  used  to  address 
future  force  planning  issues.  WAR  BREAKER  has  demonstrated  how  Army,  Air  Force,  and 
Navy  sensors,  command,  control,  communications,  and  intelligence  (C3I),  and  weapons 
systems  could  be  teamed  to  attack  critical  mobile  targets.  In  this  demonstration,  existing 
systems  were  considered  to  test  the  feasibility  of  the  simulation  network  itself.  But  the  dem¬ 
onstrations  suggests  how,  in  the  future,  different  mixes  of  current  and  proposed  systems, 
along  with  new  tactics  and  doctrine,  could  be  explored  in  the  simulation  environment. 
Analyses  of  this  kind  bridge  the  gap  between  the  concept  development  community  and  the 
community  of  force  planners  and  budgeters. 

WAR  BREAKER’S  distributed  simulation  network  plows  new  ground  in  permitting 
the  military  to  explore  combat  approaches  involving  seamless,  multi-Service,  “system  of 
systems”  approaches  for  performing  future  military  tasks.  Modernization  planning  will 
increasingly  have  to  address  these  complex  “system  of  system”  issues  because  limited  DoD 
modernization  budgets  will  have  to  focus  on  finding  investments  that  enhance  the  capabil¬ 
ity  of  current  platforms.  The  role  of  distributed  simulation  in  force  modernization  planning 
is  thus  likely  to  expand  as  rapidly  as  the  simulation  technologies  permit  it. 

Complementing  these  force-level  combat  simulations  are  force  costing  models  such 
as  the  Force  Acquisition  Cost  System  (FACS).  These  allow  DoD  resource  planners  to  esti- 
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mate  total  program  costs  and  budget  requirements  for  force  options.  Costs  include  research 
and  development,  acquisition,  and  operations  and  maintenance.  When  such  models  are 
combined  with  force  effectiveness  evaluation  models,  such  as  TACWAR,  additional 
options  for  program  and  force  structures  can  be  considered.  Such  assessments  provide  use¬ 
ful  support  in  evaluating  broad  tradeoffs  in  designing  modernization  programs. 

Force  planning  and  costing  simulations  could  offer  an  important  bridge  between  the 
communities  involved  in  requirements  setting,  force  planning,  and  budgeting.  The  acquisi¬ 
tion  community  can  play  a  significant  role  by  ensuring  that  future  models  and  simulations 
are  compatible  across  these  communities.  We  will  return  to  this  question  when  we  take  up 
the  issue  of  how  simulation  is  supporting  cross-functional  integration. 

6.  Program  Management 

Traditionally,  program  management  simulations  of  costs  and  schedules  have  been 
an  important  tool  for  planning  and  executing  major  weapon  programs.  Cost  modeling  has, 
of  course,  been  done  extensively  within  the  DoD  for  decades.  Parametric  cost  models, 
based  on  historical  statistical  relationships,  provide  estimates  of  likely  program  costs. 

Similarly,  program  planning  and  process  models  have  long  been  used  to  structure 
and  sequence  program  tasks.  Their  use  stems  back  to  the  creation  of  the  Project  Evaluation 
and  Review  Technique  (PERT)  developed  for  the  Navy’s  Polaris  program.  One  recent 
application  of  process  modeling  is  in  an  ARPA  program  which  studied  the  Vertical  Take¬ 
off  and  Landing  -  Unmanned  Aerial  Vehicle  (VTOL-UAV)^  program.  The  FORESIGHT 
process  modeling  tool,  an  extension  of  IDEF^  concepts,  was  used  to  help  structure  the  tasks 
required  to  conduct  a  planned  exercise,  as  well  as  to  surface  issues  and  problems  that  might 
not  otherwise  have  been  anticipated. 

There  are  many  new  management  practices  that  have  been  developed  and  applied 
in  various  levels  of  detail  within  several  programs.  The  F-22  program  is  representative  in 
its  attempts  to  include  the  key  features  of  concurrent  engineering  and  TQM  concepts.  They 
have  organized  cross-functional  teams,  have  made  the  industry  contractors  an  integral  part 

^  Previously  called  the  Maritime  Unmanned  Aerial  Vehicle  (MUAV). 

*  The  acronym  IDEF  originally  stood  for  the  ICAM  [definition  language.  ICAM  (Integrated-Computer  Aid¬ 
ed  Manufacturing)  was  the  sponsoring  organization  within  the  U.S.  Air  Force  for  the  original  IDEF  devel¬ 
opment  efforts.  More  recently,  the  Information  Integration  for  Concurrent  Engineering  (IICE)  Program  has 
assumed  responsibility  for  the  continued  development  of  the  advanced  methods  of  IDEF.  The  IICE  Pro¬ 
gram  now  refers  to  IDEF  as  the  Integrated  Definition  Language  [Rupp  1993,  p.  328]. 


of  the  program  decision-making  process,  and  have  developed  measures  of  performance  for 
many  key  activities  [Winner  1993]. 

These  cost  and  schedule  simulation  tools  represent  an  interesting  extension  to  the 
current  parametric  approaches.  On  the  one  hand,  the  use  of  parametric  cost  models  and  pro¬ 
gram  planning  simulations  are  among  the  most  mature  applications  of  modeling  and  simu¬ 
lation  among  those  in  use  today.  But,  on  the  other  hand,  the  application  of  these  tools  will 
be  subject  to  the  greatest  change  as  new  development  practices  are  adopted.  Whereas  these 
tools  have  traditionally  been  used  to  extrapolate  costs  and  schedules  from  past  experience, 
in  the  future  they  will  be  used  as  a  guide  to  re-engineering  the  development  process.  New 
process  simulation  tools  will  permit  managers  to  consider  optional  structures  for  programs. 
The  results  of  one  of  the  DARPA  Initiative  on  Concurrent  Engineering  (DICE)  programs 
provide  an  illustration  of  how  process  modeling  could  be  used  to  restructure  the  develop¬ 
ment  process  to  perform  tasks  more  effectively  [Singh  1992,  pp.  14-23].  One  effort  within 
GE  Aircraft  Engines  reduced  the  development  cycle  time  from  18  months  to  7.5  months 
while  at  the  same  time  increased  the  number  of  design  iterations  (improving  the  quality  of 
the  design)  from  1 1  to  over  60.  In  time,  such  process  models  should  become  a  routine  tool 
for  implementing  concurrent  engineering  practices  in  DoD  programs. 

7.  Conclusions  Based  on  Functional  Review 

The  examples  presented  in  each  of  the  functional  communities  demonstrate  the 
wide  range  of  applications  of  simulation  in  the  defense  acquisition  process.  In  one  form  or 
another,  simulation  is  used  by  every  one  of  these  communities.  The  maturity  of  the  appli¬ 
cations  varies  substantially  across  functional  communities,  across  simulation  families,  and 
within  the  functional  communities.  In  part,  this  reflects  the  fact  that  engineering  simula¬ 
tions  have  largely  been  developed  in  the  private  sector,  and  the  other  functional  simulation 
technologies  have  largely  been  developed  by  the  communities  that  use  them.  There  has 
been  little  incentive  for  establishing  commonality  in  modeling  and  simulations  designs  or 
data  formats. 

Advances  in  technology,  data,  and  applications  will  occur  in  every  functional  com¬ 
munity,  so  the  contribution  of  simulation  will  grow.  These  advances  suggest  that  simulation 
will  contribute  to  improvements  in  the  learning  and  convergence  process  within  functional 
communities,  independent  of  DoD’s  investment  or  in  its  progress  in  adapting  commercial 
development  approaches.  However,  as  the  commercial  experiences  reviewed  in  Section  II 
have  suggested,  the  use  of  new  management  practices  and  the  utilization  of  simulations 
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could  make  a  significant  contribution  if  it  were  used  to  support  integrating  activities  across 
the  function  communities.  Such  applications  are  discussed  in  the  next  two  subsections. 

B.  CROSS-FUNCTIONAL  DISTRIBUTED  SIMULATION 

We  noted  in  the  introduction  to  Section  ID  that  the  simulation  tools  used  in  defense 
acquisition  fall  into  four  families:  combat,  physical,  availability  and  sup’'ort,  and  cost  and 
schedule.  This  subsection  examines  the  degree  to  which  there  is  common  uses  of  simula¬ 
tions  within  similar  families  and  across  the  functional  communities.  Such  commonality 
could  be  the  basis  for  integrating  future  development  activities  within  those  communities. 

We  find  there  is  considerable  overlap  across  communities  in  the  use  of  simulation, 
suggesting  that  simulation  tools  could  provide  a  common  ground  among  communities.  We 
have  identified  where  each  family  of  simulation  is  used  by  nearly  all  of  the  six  acquisition 
functional  communities,  and  each  community  uses  nearly  all  of  the  families  of  tools  (Table 
III-3).  In  practice,  however,  the  degree  of  integration  afforded  by  this  overlap  is  limited  by 
the  lack  of  compatibility  among  simulation  tools  within  each  family.  Some  examples  are 
presented  in  this  section  to  illustrate  the  current  state  of  the  art. 

1.  Combat 

The  first  row  of  the  table  summarizes  the  various  applications  of  combat  simula¬ 
tions  in  the  su  functional  communities.  The  McDonnell  Douglas  domed  flight  simulator 
facility  provides  an  excellent  illustration  of  how  a  simulator  could  be  used  in  the  first,  sec¬ 
ond,  and  third  communities.  First,  the  AMRAAM  simulations  described  earlier  illustrate 
the  use  of  that  facility  in  defining  concepts  and  setting  requirements.  The  domes  have  also 
been  used  for  engineering  studies,  particularly  when  human  factor  aspects  are  being 
defined.  They  have  been  used  extensively  for  cockpit  design  layouts,  for  example,  and  pres¬ 
ently  are  being  used  to  evaluate  different  configurations  of  the  MIDS  aircraft  data  terminal 
for  the  F-18  aircraft. 

These  simulators  have  also  been  used  to  design  and  augment  operational  tests,  for 
example,  in  the  extensive  use  of  the  simulators  during  the  F-15  operational  tests.  McDon¬ 
nell’s  domed  simulators  generally  have  not  played  directly  in  force  planning  analyses 
because  they  assess  few-on-few,  rather  than  force-on-force  engagements,  and  thus  do  not 
capture  the  breadth  of  engagement  needed  for  force  planning.  However,  the  WAR  BREAK¬ 
ER  demonstration  cited  earlier  provides  an  interesting  counterexample  because  it  incorpo¬ 
rated  McDonnell’s  F-15  simulator  in  a  SIMNET  distributed  network  simulation  to  assess 
system  compatibility,  command  and  control  operations,  and  system  performance.  In  sum- 
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Table  II1-3.  Examples  of  Common  Simulations  Used  Within  Each  Simulation  Family 

and  Functional  Community 


Family  of 
Simulation 

Uses  by  Functional  Community 

Require¬ 
ments  & 
Concept 
Develop¬ 
ment 

Engineer¬ 
ing  Design 
&  Manu¬ 
facturing 

Test  & 
Evaluation 

Logistics 
&  Support 

Force 
Planning 
&  Budget¬ 
ing 

Pn^ram 

Manage¬ 

ment 

Combat 

Require¬ 
ments  & 
Concepts 

Human 

Factors 

Operational 
Test  Design 

Support 

Alternatives 

Force 

Planning 

Organiza¬ 

tional 

Structure 

Physical 

System  & 

Subsystem 

Trade-offs 

Detailed 
Design  & 
Mfg.  Pro¬ 
cess  Design 

Develop¬ 
mental  Test 
Design 

Reliability 
&  Maintain¬ 
ability 

Assess¬ 

ments 

Availability 
&  Support 

Availability 
of  Systems 
&  SuppUes 

Design 

Trade-offs 

Develop¬ 
mental  Test; 
Operational 
Test 

Logistics 

Planning 

Contingen¬ 
cy  and 
Operational 
Needs 

Organiza¬ 

tional 

Structure 

Cost  & 
Schedule 

Cost  &  Per¬ 
formance 
Trade-offs 

Cost  &  Per¬ 
formance 
Trade-offs 

Cost  &  Per¬ 
formance 
Trade-offs 

Budget 

Planning 

Cost  & 
Schedule 
Planning 
Manage¬ 
ment 

mary,  there  is  a  high  degree  of  overlap  of  interests  across  functional  communities  in  the  use 
of  a  combat  simulation  capability  such  as  exists  for  the  F-15. 

The  WAR  BREAKER  example  points  to  the  earlier  observation  that  there  are  sig¬ 
nificant  differences  across  communities  in  the  type  and  the  level  of  detail  of  the  simulation 
tools  used.  For  example,  the  detail  required  to  model  and  simulate  theater-level  combat 
support  force  planning  is  quite  different  from  the  detail  required  to  support  the  development 
of  a  manned  weapon  system  or  to  study  the  effectiveness  of  a  sensor  concept  within  a  com¬ 
plex  set  of  environmental  conditions  and  scenario  variations.  Moreover,  existing  simulators 
employ  different  software  and  data.  A  major  challenge  of  the  WAR  BREAKER  demonstra¬ 
tion  was  simply  to  connect  diverse  simulators.  While  each  community  may  have  a  different 
set  of  expectations  of  the  level  of  detail  or  the  type  of  simulation  required  for  its  application, 
a  common  basis  should  be  sought  for  developing  and  using  simulations  within  a  given  fam¬ 
ily.  As  simulation  technology  advances,  it  should  be  used  to  provide  a  common  basis  for 
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the  analyses  conducted  within  each  community,  and  to  provide  an  effective  channel  of  com¬ 
munication  among  the  communities  employing  combat  simulation. 

2.  Physical 

Whereas  the  domed  simulators  provide  a  common  simulation  medium  across  sev¬ 
eral  communities,  there  is  a  wide  range  of  types  of  physical  simulators  used  in  acquisition 
programs.  Recognizing  this,  the  engineering  community  has  initiated  efforts  to  provide  a 
common  language  that  will  integrate  these  physical  simulators.  Such  activities  will  serve 
to  increase  integration  within  the  engineering  and  manufacturing  community,  as  well  as  to 
include  logistic  support  issues  within  the  development  process.  Foremost  among  these  is 
the  support  of  the  Departments  of  Defense  and  Commerce  for  a  product  data  exchange 
standard  (PDFS)  which  would  allow  commercial  development  tools  and  simulators  to 
exchange  information.  A  universal  language  for  engineering  design  tools  could  allow  any 
of  the  tools  within  the  engineering,  manufacturing,  and  support  families  of  simulation  to 
work  together.  This  would  greatly  facilitate  the  creation  of  integrated  design  teams  involv¬ 
ing  members  that  formerly  were  separated  by  their  use  of  disparate  engineering  tools. 

The  integration  of  logistics,  support,  and  analysis  data  bases  with  design  and  engi¬ 
neering  data  bases  has,  of  course,  been  an  explicit  goal  of  the  DoD’s  CALS  program  since 
the  mid-1980s.  New  acquisition  programs  are  required  to  meet  CALS  data  exchange  stan¬ 
dards,  and  this  is  helping  to  create  the  needed  bridge.  The  RAMCAD  program  is  also  striv¬ 
ing  to  integrate  support  considerations  within  design  and  engineering  tools.  A  key  limiting 
aspect  on  integration  across  organizations  within  this  family  are  proprietary  and  personnel 
sensitivity  concerns.  Appropriate  security  safeguards  are  needed  to  help  ensure  vested 
ideas,  concepts,  and  limitations  are  adequately  protected. 

3.  Availability  and  Support 

The  concern  for  weapon  system  availability  and  support  has  led  to  the  development 
of  a  large  number  of  models  and  simulations.  The  requirements  and  concept  development 
community  has  used  these  models  to  help  define  specifications  and  evaluate  new  design 
concepts  and  approaches  and  to  plan  for  a  weapon’s  logistic  support.  The  engineering 
design  and  manufacturing  communities  use  these  types  of  models  and  simulations  as  they 
iterate  on  design,  producibility,  and  support  issues. 

Detailed  component  studies  of  electronic  parts  and  mechanical  interactions  have 
been  the  basis  for  many  engineering  and  manufacturing  efforts.  Many  simulations  model 


the  mean-time-between-failure  and  mean-time-to-repair  of  both  mechanical  and  electronic 
components.  The  RAMCAD  efforts  mentioned  earlier  support  these  and  other  analyses. 

The  LCAM  gives  estimates  of  the  logistics  support  needed  for  new  aircraft.  Similar 
simulations  give  insights  to  the  needs  for  ground  vehicle  and  water  craft. 

4.  Cost  and  Schedule 

Presently,  cost  and  schedule  models  and  simulations  remain  largely  unconnected 
with  the  other  families  of  simulators.  In  the  future,  however,  the  ability  to  simulate  the 
development  process  using  IDEF,  or  other  similar  tools,  should  integrate  costs  and  schedule 
estimation  with  the  engineering  development  tasks,  thus  permitting  more  accurate  esti¬ 
mates  to  be  made  of  costs,  schedules,  and  risks. 

The  needed  linkage  between  engineering,  manufacturing,  and  support  simulation 
and  cost  estimation  is  being  forged  through  the  development  of  activity-based  cost  (ABC) 
accounting  [Drucker  1990,  p.  97].^  ABC  focuses  on  allocating  both  direct  and  overhead 
costs  to  identifiable  activities,  rather  than  simply  treat  overhead  as  a  loading  factor  on  direct 
labor  hours.  Allocating  costs  to  these  activities  gives  management  a  clearer,  and  very  dif¬ 
ferent,  view  of  costs  than  is  provided  by  conventional  cost-allocation  methods  [Cooper 
1991,  p.  132].  In  time,  the  emerging  design  and  engineering  tools  wUl  provide  the  activity- 
based  data  needed  to  estimate  costs  using  ABC  principles,  and  this  in  turn,  will  provide  bet¬ 
ter  estimates  of  costs  as  well  as  better  insights  into  the  design  of  manufacturing  processes 
[Dilts  1990,  pp.  50-53]. 

In  summary,  limited  capabilities  presently  exist  for  using  simulation  to  communi¬ 
cate  across  functional  boundaries.  These  communication  “bridges”  operate  mainly  in  two 
areas:  first,  in  programs  where  a  common  combat  simulation  can  be  used  for  requirements 
determination,  operational  test  planning,  and  force  planning;  and  second,  where  there  are 
programs  whose  engineering  data  are  used  in  manufacturing  and  support  communities. 
Substantial  efforts  already  underway  further  break  down  the  barriers  within  and  across  fam¬ 
ilies  of  simulation.  These  efforts  should  be  expanded  and  include  the  development  of  an 
overarching  architecture  for  the  functional  communities  and/or  family  areas.  Particular 
emphasis  should  be  placed  on  defining  the  interfaces  between  the  functional  communities 

^  Peter  Drucker  observes  that  current  accounting  practices  evolved  earlier  this  century,  when  direct  blue  col¬ 
lar  labor  accounted  for  80%  of  all  direct  manufacturing  costs.  Drucker  notes  that  a  consortium  of  automat¬ 
ed  equipment  manufacturers  has  spearheaded  the  development  of  new  cost  accounting  methods,  because 
traditional  methods  could  not  adequately  measure  the  benefits  of  investments  in  new  manufacturing  tech¬ 
nologies.  This  consortium  is  Computer-Aided  Manufacturing-International,  CAM-I. 
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for  each  family.  These  need  continued  support  from  the  DoD.  The  DoD  should  work  to  also 
build  bridges  among  the  DoD’s  simulation  families,  and  between  them  and  the  commer¬ 
cially  developed  design  tools  and  simulations.  Tliis  is  an  area  where  the  defense  acquisition 
community  has  a  central  responsibility,  and  where  progress  is  unlikely  to  occur  without 
support  from  senior  DoD  leadership. 

It  will  be  necessary  to  bridge  the  very  detailed  descriptive  and  physical  performance 
data  that  exists  in  the  engineering  environment  with  the  more  macro  performance  data  that 
are  used  as  inputs  in  combat  or  cost  and  schedule  simulations.  Because  of  the  complexity 
of  this  task,  it  will  probably  be  many  years  before  automated  data  exchanges  among  these 
families  of  simulation  will  be  possible. 

C.  THE  VISION:  A  FUNCTIONALLY  INTEGRATED  SYSTEM  ACQUISI¬ 
TION  PROCESS 

The  ultimate  goal  of  efforts  to  integrate  development  tools  is  to  extend  simulation 
capabilities — visualization,  performance  modeling,  common  data,  and  user  interaction — 
seamlessly  across  functional  boundaries.  A  fully  integrated  simulation  system  would  pro¬ 
vide  each  community  access  to  a  common  data  base,  and  allow  each  community  to  visual¬ 
ize  and  understand  the  design  and  its  capabilities  from  its  own  perspective.  For  example, 
concept  designers  could  view  a  design  in  a  simulated  combat  environment.  Testers  would 
see  it  in  a  lab  or  on  a  test  range.  Manufacturers  could  see  it  in  the  context  of  a  simulated 
manufacturing  system.  Logisticians  could  see  it  in  terms  of  reliability  statistics,  individual 
repair  tasks  or  unit-level  support  requirements,  or  theater  logistics  flows.  Full  integration 
of  distributed  simulation  capabilities  should  substantially  reduce  the  communications  bar¬ 
riers  among  acquisition  functional  communities.  It  should  also  improve  risk  management, 
by  allowing  each  community  to  visualize  a  design  in  the  context  that  best  allows  them  to 
anticipate  issues  and  problems. 

Functionally  integrated  simulation  capabilities  will  help  fulfill  the  vision  of  “agile 
manufacturing”  and  “agile  logistics.”  Agility  refers  to  the  ability  of  suppliers  to  rapidly  and 
cost  effectively  respond  to  customer’s  needs.*®  Agility  incorporates  five  main  elements 
which  rely  heavily  on  distributed  information  technologies: 

•  Electronic  links  among  customers,  designers,  and  suppliers. 


Here  we  use  supplier  and  customer  in  the  broadest  context  and  as  used  in  TQM  applications.  Also  see 
[Uhigh  1991;  Port  1992]. 


•  Electronic  data  interchange  (EDI)  among  suppliers,  allowing  design  and  engi¬ 
neering  data  to  flow  among  them. 

•  “Vutual  companies”  consisting  of  teams  of  key  marketers,  designers,  and  sup¬ 
pliers  from  diverse  organizations  that  can  be  formed  and  dissolved  as  needed  to 
meet  market  demands. 

•  Flexible  manufacturing  allowing  each  supplier  to  rapidly  adapt  its  product  line 
to  satisfy  customer  demands. 

•  Express  distribution  allowing  components  to  flow  quickly  among  suppliers  and 
finished  products  to  quickly  reach  customers. 

The  long-run  vision  for  an  agile  manufacturing  sector  would  include  an  information 
“highway”  that  could  be  accessed  by  customers,  designers,  and  suppliers;  engineering  data 
bases  to  support  rapid  product  development:  and  common  data  standards  and  languages 
that  would  permit  easy  communication.  The  role  for  distributed  simulation  technologies  in 
this  concept  is  to  enhance  communication  across  organizations  with  different  functional 
perspectives.  It  provides  the  needed  common  language  and  visualization  that  would  allow 
the  interconnected  entities  to  communicate  effectively.  This  vision  provides  a  starting  point 
for  defining  the  DoD’s  long-range  objectives  for  simulation-related  programs  and  policies 
and  achieving  a  functionally  integrated  system  acquisition  process. 

D.  CONCLUSIONS  OF  FUNCTIONAL  AND  CROSS-FUNCTIONAL 

REVIEW 

Our  review  shows  that  simulation  tools  and  applications  are  quite  advanced  in  some 
application  areas,  as  for  example  in  the  use  of  domed  flight  simulators  for  aircraft  engineer¬ 
ing  and  manufacturing  development,  and  testing  and  evaluation.  Computer-aided  design 
and  engineering  tools  are  also  rapidly  developing,  allowing  a  range  of  components  to  be 
represented  and  their  physical  characteristics  modeled.  The  underlying  base  of  detail 
design,  environment  and  scenario  data,  behavioral  models,  and  model  validation  proce¬ 
dures  remains  sparse,  but  will  grow  as  new  applications  are  undertaken. 

Thus  distributed  simulation  is  becoming  a  reliable  tool  for  developers  in  many 
applications.  However,  it  is  only  beginning  to  provide  capabilities  for  communication 
across  functional  areas.  We  conclude,  therefore,  that  considerable  further  development  of 
simulation  tools,  with  a  focus  on  cross-functional  integration,  will  be  required  before  dis¬ 
tributed  simulation  can  fulfill  its  promise  of  supporting  an  integrated  acquisition  process. 
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IV.  TOWARD  AN  AGENDA  FOR  APPLYING  DISTRIBUTED 
SIMULATION  IN  THE  ACQUISITION  PROCESS 


Commercial  experience  suggests  that  improvements  in  the  DoD’s  acquisition  pro¬ 
cess  will  require  a  combination  of  both  investments  in  simulation  capabilities  and  imple¬ 
mentation  of  management  initiatives  aimed  at  establishing  a  functionally  integrated 
acquisition  process.  This  section  describes  the  building  blocks  of  a  strategy  in  each  of  these 
two  areas.  The  first  subsection  examines  the  investments  needed  to  improve  and  expand 
distributed  simulation  capabilities.  The  second  explores  the  kinds  of  management  changes 
needed.  These  two  areas  should  form  the  core  of  DoD’s  efforts  in  applying  simulation  tech¬ 
nology  to  improve  the  development  process,  and  represent  the  main  focus  of  the  discussion 
in  this  section.  There  are,  in  addition,  a  number  of  other  acquisition  reform  initiatives — ^par¬ 
ticularly  “prototyping  plus”  and  “dual  use” — that  can  also  benefit  from  simulation  capabil¬ 
ities.  We  discuss  these  at  the  conclusion  of  this  section. 

A.  ELEMENTS  OF  AN  INVESTMENT  STRATEGY 

Realizing  the  full  potential  of  simulation  in  the  defense  acquisition  process  will 
require  large  investments  over  many  years.  The  purpose  of  this  section  is  to  summarize  the 
areas  where  investments  will  be  needed.  We  describe  six  main  building  blocks  that  should 
compose  an  investment  strategy  for  simulation,  i.e.,  simulation  hardware  and  software; 
application  tools;  communication  standards  and  protocols;  behavioral  models  and  data; 
VV&A;  and  education  and  training  (Table  FV-D  These  building  blocks  form  the  general 
outline  that  can  be  fashioned  into  a  fully  developed  strategy  by  identifying  specific  projects 
within  each  area  and  appropriately  balancing  resources  across  areas.  The  priorities  within 
this  strategy  should  reflect  the  current  state  of  the  art  and  ongoing  development  activities 
in  each  area,  and  allocate  resources  appropriately  to  achieve  balance  in  progress  across 
capabilities.  Each  area  is  discussed  in  turn. 


Table  IV-1.  Investment  Strategy  Building  Blocks 


Investment  Area 

Main  Developmental  Funding  Source 

1.  Hardware  and  Software  Supporting 
Distributed  Simulation  Networks 

Commercial 

2.  DoD  Application  Tools 

•  Simulation  Constmction  Software 

•  Manned  Combat  Simulators 

•  Semi-automated  Forces 

•  Weapon  Production  Process 

•  Simulators 

DoD 

3.  Cross-Functional  Standards,  Protocols, 
and  Bridges 

DoD.  DOC,  and  Commercial 

4.  Underlying  Behavioral  Models,  Data 
Bases,  Data  Capture  Instmmentation, 
Data  Reduction  and  Analysis  Software 

•  Combat 

•  Engineering  and  Manufacturing 

•  Support 

•  Cost  and  Schedule  Mgmt. 

•  DoD 

•  DoD  and  Commercial 

•  DoD 

•  DoD  and  Commercial 

5.  Verification,  Validation  and  Accredita¬ 
tion  (W&A): 

•  Combat 

•  Engineering  and  Manufacturing 

•  Support 

•  Cost  and  Schedule  Mgmt. 

•  DoD 

•  Commercial  and  DoD 

•  DoD 

•  DoD  and  Commercial 

6.  Education  and  Training 

DoD  and  Commercial 

1.  Investment  Strategy  Building  Blocks 

The  first  building  block  comprises  basic  computer  hardware  and  software,  includ¬ 
ing  networking  capabilities.  The  highly  competitive  commercial  technology  markets  are 
providing  rapid  increases  in  computer  speed  and  capacity,  graphics  capabilities,  and  data 
network  capacity.  A  recent  Defense  Science  Board  study  of  advanced  distributed  simula¬ 
tion  concluded  that  these  commercial  developers  would  drive  advances  in  these  generic 
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simulation  technologies  [DSB  Sim  1993,  Appendix  A].  This  area  represents  an  important 
example  of  dual-use  technology.  The  DoD  should  rely  on  these  commercial  developments 
and  focus  its  efforts  on  adapting  them  as  needed  for  defense  applications.  An  even  more 
activist  approach  would  be  to  support  basic  research.  For  example,  the  DoD  could  suppon 
research  on  flat  panel  displays  which  will  have  significant  civilian  and  defense  applications. 
The  DSB  also  prepared  a  table  of  technologies  and  the  major  sector  which  drives  each  tech¬ 
nology.  That  table  is  given  in  Table  IV-2. 


Table  IV>2.  Key  M&S  Technologies  and  Source  of  Major  Support^ 


Areas  Driven  by 
Commercial  Market 

Technologies  Driven  by  DoD 

Integrated  Circuits 

Micro  computer  systems 

Local-area  networks 

Fiber-optic  communications 
Wide-area  networks 

Computer  image  generators 

High  performance  computing 
Memory 

Mass  storage 

Microprocessors 

High  resolution  video  displays 

Data  base  management  systems 
Software  engineering  tools 

Manufacturing  process  simulators 
Engineering  design  modeling  and 
simulation 

Manned  simulators 

Stochastic  wargaming  simulators 
SAFOR  (Semi- Automated  Forces) 
Instrumented  range  systems 
Environmental  models/data  bases 
Human  representation  models 

Human  ergonomics  data  bases 
Instrumented  range  systems 

DoD  protocols  and  standards 
Multi-level  security  techniques 
Modeling  and  simulation  construction 
tools 

a.  [DSB  Sim  1993] 


Even  if  the  DoD  does  not  support  generic  simulation  technologies,  there  is  a  wide 
range  of  application  tools  specific  to  DoD’s  needs  that  need  to  be  developed.  These  com¬ 
pose  the  second  building  block.  Included  are  combat  applications  of  generic  technologies. 
The  DoD  needs  to  develop  the  manned  simulators,  combat  environments,  and  automated 
forces  needed  for  applying  simulation  to  the  combat  environment.  Investments  will  be 
needed  in  the  engineering  and  manufacturing  communities  as  well.  These  primarily  relate 
to  materials  or  manufacturing  processes  that  are  unique  to  military  items,  such  as  stealth 
manufacturing  technologies.  Most  of  the  DoD  investments  in  simulation  technologies  to 
date  have  concentrated  in  these  types  of  application-specific  tools. 


Hie  third  building  block  includes  the  data  standards  and  bridges  needed  to  connect 
the  simulation  effort  within  the  various  functional  communities.  Many  years  of  investment 
will  be  required  before  it  will  be  possible  to  freely  exchange  data  across  functional  bound¬ 
aries,  but  if  this  vision  is  to  be  realized,  the  DoD  must  begin  to  take  steps  in  that  direction 
today.  An  important  focus  of  this  effort  is  extensive  interaction  with  commercial  developers 
to  create  the  architectures  and  appropriate  bridges  needed  within  and  between  commercial 
and  DoD-specialized  families  of  simulation.  Hie  goal  is  for  key  personnel  within  each  of 
the  functional  communities  to  exchange  information  via  these  interfaces  with  each  of  the 
other  communities.  Such  information  would  include  descriptions  of  the  systems,  associated 
environments,  and  scenarios  and  performance  data  such  as  the  capabilities  and  limitations 
of  the  proposed  designs  within  the  stated  environmental  conditions  and  scenarios.  These 
data  should  be  available  with  variable  resolution,  fidelity,  and  aggregation  as  needed  by 
personnel  in  each  functional  community. 

Although  there  is  a  tendency  to  focus  on  the  technical  challenges  of  building  the 
simulation  networks  themselves,  the  bedrock  upon  which  a  simulation  network  must  be 
built  is  its  performance  models  and  data  bases.  These  make  up  the  fourth  building  block  of 
a  simulation  investment  strategy.  The  development  of  the  underlying  models,  data,  and  ana¬ 
lytical  tools  needed  for  the  combat  and  support  families  of  simulation  must  come  from  the 
DoD  because  these  tend  to  be  specialized.  The  models  and  data  for  engineering  and  manu¬ 
facturing,  and  management  tend  to  be  dual  use  technologies,  so  DoD  and  commercial  sup¬ 
port  is  available.  Extensive  investments  will  be  required  over  many  years  to  build  the 
needed  library  of  models  and  data  needed  to  fully  realize  the  potential  of  simulation  in 
defense  acquisition.  The  need  for  such  a  library — along  with  a  user  guide  to  facilitate  ready 
access  for  future  developers — should  be  recognized  as  a  key  part  of  the  overall  investment 
strategy  for  simulation. 

The  fifth  building  block,  verification,  validation,  and  accreditation  (W&A),  is  one 
of  the  most  sensitive  issues  in  dealing  with  models  and  simulations.  It  cuts  across  all  fam¬ 
ilies  of  simulation  and  affects  each  of  the  functional  communities.  The  credibility  of  any 
model  or  simulation  rests  on  the  adequacy  of  the  models  or  their  data  bases  to  represent  the 
real  world.  Each  model  must  be  verified  to  ensure  that  it  performs  as  specified  in  the 
approved  design.  Verification  turns  out  to  be  a  matter  of  degree  for  most  complex  models, 
because  it  is  impossible,  in  practice,  to  test  these  models  over  the  entire  range  of  variable 
values.  In  addition,  it  is  often  not  feasible  with  the  available  resources  to  do  a  line  by  line 


check  of  the  code.  Proper  processes  need  to  be  established  and  utilized  by  organizations  to 
ensure  a  simulation  can  be  verified. 

Validation,  on  the  other  hand,  must  ensure  that  the  model  (and  its  data)  predicts, 
within  specified  levels  of  confidence,  the  behavior  of  the  real  system  being  modeled.  A 
model  may  be  characterized  as  being  valid  in  several  ways  [Davis  1992].  It  may  have 
descriptive,  structural,  or  predictive  validity.  “Descriptive  validity”  means  that  the  model 
is  able  to  explain  phenomena  or  organize  information  meaningfully  in  one  way  or  another. 
A  model  has  “structural  validity”  if  it  has  the  appropriate  entities,  attributes,  and  processes 
so  that  it  corresponds  well  with  the  real  world.  “Predictive  validity”  means  that  a  model  can 
predict  desired  features  of  system  behavior  to  within  some  known  level  of  accuracy  and 
precision.  Predictive  validity  relates  to  the  issue  of  extrapolation  error  raised  earlier.  To 
enhance  the  validation  of  a  model,  people  from  each  functional  community  and  with 
diverse  experiences  may  need  to  lend  their  observations  and  concerns.  This  is  particularly 
needed  when  a  system’s  behavior  is  to  be  estimated  in  a  large  set  of  environments  and  sce¬ 
narios. 

Finally  each  simulation  must  be  certified  or  accredited  according  to  uniform  criteria 
that  clearly  state  the  capabilities  and  limitations  of  the  simulation,  the  interface  require¬ 
ments  for  linking  the  simulation  with  other  simulations,  the  environment  within  which  the 
system  simulation  is  to  operate,  and  the  scenarios  describing  how  the  system  being  mod¬ 
eled  is  to  be  used.  In  effect,  accreditation  is  an  official  determination  that  a  simulation  is 
acceptable  and  for  which  specific  purposes  or  classes  of  applications. 

There  is  another  issue  which  is  related  to  VV&A — the  repeatability  of  results.  Espe¬ 
cially  in  the  use  of  manned  distributed  simulations,  the  expense  and  complexity  of  such 
simulations  may  not  permit  sufficient  data  to  be  taken  with  enough  diverse  personnel  to 
gain  sufficient  statistically  significant  information.  Reconnections  of  the  distributed  simu¬ 
lations  may  also  be  difficult  and  thus  complete  repeatability  may  not  be  possible. 

The  sixth  investment  building  block  is  the  education  and  training  of  acquisition  per¬ 
sonnel  in  the  application  of  modeling  and  simulation.  The  SEMNET  activities  have  greatly 
facilitated  the  education  and  training  of  both  helicopter  and  tank  personnel.  Extensions  to 
other  functional  communities  have  not  yet  been  given  priority.  In  time,  these  tools  will 
become  standard  offerings  in  the  training  of  personnel  in  all  functional  communities  and 
other  phases  of  acquisition.  Currently  there  is  little  organized  effort  directed  in  this  critical 
area. 
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2.  Funding 

The  recent  Defense  Science  Board  report  on  modeling  and  simulation  identifies 
several  major  programs  currently  funding  investments  in  simulation  [DSB  Sim  1993]. 
ARPA  and  the  Defense  Modeling  and  Simulation  Office  (DMSO)  together  provide  about 
$110  million  for  simulation-related  research.  In  addition,  each  of  the  Services  is  funding 
specific  application  projects.  Currently,  the  DMSO,  which  supports  the  Executive  Council 
for  Modeling  and  Simulations  (EXCIMS),  and  the  Steering  Group,  which  is  helping  the 
Science  and  Technology  Thrust  6,  “Synthetic  Environments,”  are  the  two  main  entities 
reviewing  ongoing  programs.  Both  will  influence  plans  to  set  DoD  priorities  for  future  sim¬ 
ulation  investments.  Such  a  process  should  help  provide  the  needed  balance  across  the 
areas  discussed. 

Other  important  funding  sources  for  creating  the  needed  simulation  building  blocks 
are  ongoing  acquisition  programs  and  demonstration  programs  of  selected  technology 
applications  and  those  of  simulation  technologies  and  management  practices.  Numerous 
ongoing  applications  of  functional  simulation  were  described  earlier,  but  in  addition  there 
are  a  number  of  demonstration  projects  proposed  or  underway. 

The  current  and  proposed  program  of  the  Science  and  Technology  Thrust  7,  Tech¬ 
nology  for  Affordability,  includes  a  series  of  demonstrations  of  manufacturing  processes 
for  major  system  components,  mostly  involving  electronics.  This  approach  has  been 
extended  to  experiment  with  complete  missile  systems,  and  could  be  extended  to  major 
subsystems  of  ships  or  aircraft,  e.g.,  an  aircraft  wing  or  a  section  of  a  ship  hull.  The  recent 
Defense  Science  Board  summer  study  on  manufacturing  has  also  proposed  a  number  of 
Advanced  Technology  Demonstration  (ATD)  programs  for  testing  new  development 
approaches.  These  include  the  Army’s  Composite  Armored  Vehicle,  the  Light  Contingency 
Vehicle,  and  the  Advanced  Field  Artillery  System.  Two  other  proposals  include  the  Multi¬ 
role  Fighter  Engine  and  the  Integrated,  High  Performance  Turbine  Engine  Technology 
(IHPTET)  [DSB  Man  1993].  Whatever  programs  are  selected  for  such  demonstrations,  the 
demonstrations  should  be  designed  to  help  create  the  needed  building  blocks  that  will  in 
turn  contribute  to  a  cumulative  base  of  simulation  capabilities. 

This  discussion  of  simulation  building  blocks  provides  a  starting  point  for  creating 
an  investment  strategy.  The  vision  of  simulation  as  a  tool  supporting  a  functionally  integrat¬ 
ed  acquisition  process  provides  a  guiding  principle  for  targeting  these  investments.  But  this 
vision  is  only  a  point  of  departure:  a  fully  developed  strategy  for  simulation  will  require  an 
extensive  effort  from  DoD  leadership,  with  help  in  the  adoption  of  the  management  prac- 
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tices  and  the  support  of  each  of  the  functional  communities  to  address  the  technical  issues. 
The  creation  of  such  a  strategy  properly  belongs  in  the  hands  of  the  communities  that  devel¬ 
op  and  use  simulation.  A  model  for  such  a  process  is  the  DoD’s  existing  Science  and  Tech¬ 
nology  Thrust  and  the  EXCIMS  processes.  TTiese  processes  bring  together  working-level 
officials  to  create  a  unified  view  and  direction  for  science  and  technology  programs.  A  sim¬ 
ilar  effort  involving  working-level  people  within  the  broader  acquisition  community,  i.e., 
government  and  industry — which  could  be  formed  under  the  sponsorship  of  the  DMSO  as 
it  has  sponsored  the  Distributed  Interactive  Simulation  (DIS)  efforts — would  form  an 
appropriate  mechanism  for  creating  a  simulation  investment  strategy. 

DMSO  has  initiated  a  task  force  which  seeks  to: 

•  Develop  and  recommend  actions  to  advance  the  effectiveness  and  efficiency  of 
the  application  of  modeling  and  simulation  throughout  the  acquisition  process. 

•  Identify  and  recommend  improvements  to  the  acquisition  process  which  could 
result  from  the  integrated  use  of  modeling  and  simulation. 

To  help  DMSO  and  this  task  force,  an  industry  group  has  been  formed.  It  too  will  develop 
its  recommendations  for  industry  and  DoD  activities. 

B.  MANAGEMENT  INITIATIVES  FOR  FUNCTIONAL  INTEGRATION 

Management  initiatives  form  the  second  area  of  the  DoD’s  agenda  which  is  needed 
to  create  a  functionally  integrated  acquisition  process.  This  section  identifies  some  of  the 
issues  involved.  We  begin  by  summarizing  the  challenges  that  have  been  faced  in  the  com¬ 
mercial  world  and  then  illustrating  the  nature  of  the  problems  that  need  to  be  addressed  in 
changing  the  acquisition  process  within  the  DoD.  We  then  focus  on  four  key  principles  that 
will  have  to  be  adopted  within  the  DoD  in  order  to  achieve  full  functional  integration  in  the 
acquisition  process.  We  conclude  thit  most  of  the  needed  changes  can  be  made  within  the 
existing  formal  acquisition  process,  but  there  are  a  few  areas  where  legal  or  regulatory 
changes  are  needed.  These  are  discussed  at  the  conclusion  of  this  section. 

1.  Lessons  from  Commercial  Experience 

We  described  earlier  how  U.S.  firms  have  improved  productivity  by  incorporating 
new  management  practices.  This  section  raises  some  issues  regarding  the  implementation 
of  these  new  approaches.  The  major  observation  that  can  be  drawn  from  the  commercial 
experience  is  that  many  of  the  firms  that  have  made  the  greatest  improvements  are  those 
that  were  driven  to  the  brink  of  bankruptcy  by  foreign  competitors  or  other  reasons.  They 


were  forced  to  innovate  or  die.  This  pattern  is  evident  among  some  of  today’s  most  progres¬ 
sive  commercial  firms. 

Xerox  is  one  well-known  example  [Kearns  1992].  During  the  late  1970s  and  early 
1980s,  Xerox’s  share  of  the  global  market  shrank  from  82  to  41  percent.  Japanese  makers 
were  selling  copiers  for  less  than  Xerox's  costs.  Xerox  realized  it  would  have  to  revitalize 
its  operations.  By  1985  it  had  cut  its  product  development  cycle  by  50  percent  and  reduced 
product  costs  by  50  percent.  Xerox  has  begun  to  regain  market  share.  Similar  stories  can  be 
told  about  some  U.S.  auto  makers.  The  Ford  Taurus  project  discussed  earlier  was  undertak¬ 
en  in  the  early  1980s  when  the  survival  of  the  company  was  in  doubt. 

Numerous  other  commercial  examples  fit  this  pattern  as  well.  John  Deere,  like 
many  other  American  firms,  was  pressed  into  radical  corporate  restructuring  by  a  combina¬ 
tion  of  stiff  foreign  competition,  labor  discontent,  and  a  slump  in  demand  for  capital  invest¬ 
ment.  Deere’s  inability  to  compete  was  due  in  large  measure  to  the  overly  complex  design 
and  production  system  that  had  evolved  when  it  was  a  predominant  supplier  of  agricultural 
equipment.  To  become  more  competitive,  Deere  initiated  cultural  changes  throughout  the 
company.  Among  other  steps,  Deere  integrated  customer  feedback  in  the  development  pro¬ 
cess  and  instituted  computer-based  support  activities.  The  results  were  remarkable.  Product 
quality  increased,  requiring  66%  fewer  inspectors  with  increased  customer  satisfaction. 
Development  costs  fell  30%,  development  time  fell  60%,  and  developmental  builds  were 
cut  from  four  to  one.  Parts  inventories  were  cut  by  65%.  Deere  continues  to  address  other 
approaches  to  become  even  more  competitive. 

Many  other  firms  that  are  cited  as  success  stories  today  were  forced  to  adapt  under 
extreme  duress.  They  include  such  leading  manufacturers  as  Caterpillar  Tractor  and  Harley 
Davidson.  In  every  case,  competition  forced  these  companies  to  adapt  to  survive.  Indeed, 
their  dire  straits  made  changes  easier  because  management  and  workers  alike  recognized 
that  the  future  of  the  company  required  dramatic  increases  in  productivity.  This  pattern  sug¬ 
gests  how  hard  it  is  to  realign  corporate  power  relationships  as  needed  to  adopt  new  prac¬ 
tices  within  the  concepts,  engineering,  manufacturing,  and  other  functional  communities. 
Traditional  stovepipe  hierarchical  structures  must  be  forced  to  work  as  teams  instead  of 
competing  power  centers. 

This  commercial  experience  underscores  the  challenges  DoD  will  face  in  imple¬ 
menting  these  new  management  practices.  New  technologies  such  as  distributed  simulation 
can  assist,  but  in  the  end  the  fundamental  challenge  is  to  create  a  development  process  with- 
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in  the  DoD  acquisition  system  that  is  compatible  with  four  key  principles  which  we  have 
identified  as  being  key  within  the  new  management  practices.  The  four  key  principles  are: 

•  Strong  leadership 

•  Functionally  integrated  development  team 

•  Clear  communication  and  expectations 

•  Simultaneous  development 

The  question  is,  how  can  these  principles  be  applied  in  the  absence  of  an  organiza¬ 
tion-threatening  crisis,  such  as  the  ones  that  forced  commercial  firms  to  adapt?  The  loss  of 
the  Soviet  threat  would  seem  to  undermine  the  pressures  for  change.  On  the  other  hand, 
budget  pressures,  combined  with  the  opportunities  created  by  the  new  distributed  simula¬ 
tion  technologies  discussed  here,  and  the  demonstrated  success  of  U.S.  commercial  firms 
in  adopting  these  new  principles,  may  be  an  adequate  catalyst  for  change.  At  a  minimum, 
such  changes  will  require  leadership  from  the  Services,  Joint  Staff,  and  the  Office  of  the 
Secretary  of  Defense. 

2.  Functional  Integration  Within  DoD  Programs 

What  will  be  required  to  implement  these  four  principles  of  the  new’  management 
practices  in  the  internal  DoD  and  its  supporting  contractor  programs?  Several  characteris¬ 
tics  of  the  approaches  suggest  the  kind  of  process  needed.  First,  however,  DoD’s  experience 
leads  us  to  add  a  fifth  principle:  effective  risk  management.  The  implementation  of  each  of 
these  five  principles  raises  management  issues  that  will  need  to  be  addressed  as  an  appro¬ 
priate  acquisition  process  is  designed  (Table  V-2). 

Strong  Leadership.  The  first  question  is,  how  to  manage  the  cross-functional  team 
participation?  Successful  commercial  developers  have  adopted  a  teaming  approach  to  cre¬ 
ate  the  program  concept.  This  approach  involves  a  strong  leader  plus  key  representatives 
from  each  of  the  major  fun,  ional  communities  which  will  be  involved  in  fielding,  operat¬ 
ing,  and  supporting  the  system.  The  challenge  is  to  apply  this  approach  within  the  DoD;  its 
breadth  must  be  expanded  beyond  the  commercial  experience  to  include  all  six  of  the 
DoD’s  acquisition  functional  communities. 
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Table  IV-3.  Adapting  the  Key  Management  Practices  to 
Defense  Acquisition  Programs 


Key  Principles  of  the  New  Man¬ 
agement  Practices 

Issue  in  DoD 
Applications 

Possible  Applications 

Strong  leadership 

How  to  incorporate  checks  and 
balances 

•  Independent  test 

•  Independent  cost 

•  Congressional  oversight 

Government-wide  participation 

•  Participate  with  support 
form  strong  leaders  finm 
all  functional  communities 

Government-industry  teaming 

Functionally  integrated  develop¬ 
ment  team 

How  to  overcome  the  traditional 
practice  of  stovepipe  develop¬ 
ment 

Create  functionally  integrated 
teams  for  ATDs  and  ACTDs 

Create  functionally  integrated 
teams  for  concept  exploration 
and  demonstration/validation 
phases  of  acquisition 

Clear  communication  and  expec¬ 
tations 

How  to  clarify  project  expecta¬ 
tions 

Document  program  performance 
metrics 

How  to  efficiently  perform  over¬ 
sight  functions 

Clarify  expectations  by  basing 
programs  on  demonstrated  tech¬ 
nologies  and  processes 

Simultaneous  development 

How  to  define/select  manufac¬ 
turing  processes  during  the 
design  development 

Include  production  options  with¬ 
in  the  engineering  design  phase 

How  to  address  testing  and  sup¬ 
port  options  during  the  design 
process 

Have  testing  and  support  person¬ 
nel  closely  involved  in  the 
design  process 

How  to  modify  performance 
requirements  to  save  cost/sched¬ 
ule 

Have  the  user  community  pre¬ 
pare  option  when  major  cost/ 
schedule  obstacles  arise 

Effective  risk  management 

How  to  overcome  the  traditional 
problem  of  entering  engineering 
and  manufacturing  development 
with  excessive  risk 

Set  clear  risk  thresholds  to 
ensure  design  convergence 
before  entering  engineering  and 
manufacturing  development 

Use  ATDs  and  ACTDs  to 
explore  all  dimensions  of  pro¬ 
gram  risk 
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Similar  integrated  approaches  have  been  suggested  before  but  found  to  be  almost 
impossible  to  fully  implement  because  the  governmental  procurement  system,  with  its 
inherent  checks  and  balances,  has  given  rise  to  a  layered,  bureaucratic  process  for  oversee¬ 
ing  programs.  “Black  programs”  have  sometimes  been  an  exception  because  such  pro¬ 
grams  typically  exclude  involvement  by  all  but  a  small  team  of  cognizant  officials  outside 
of  the  program  itself.  They  provide  one  model  for  DoD  teaming,  but  they  are  not  the  final 
answer  because  there  is  concern  that  their  checks  and  balances  are  too  weak.  A  new  hybrid 
program  structure  is  needed  that  can  implement  the  best  of  these  principles  without  unduly 
sacrificing  needed  checks  and  balances. 

Implementing  this  new  principle  also  creates  difficult  policy  issues:  how  can  inte¬ 
grated  engineering  teams  be  established  within  the  government  acquisition  environment, 
where  traditionally  these  functional  communities  have  operated  at  arm’s  length?  Is  it  pos¬ 
sible  to  create  the  kind  of  teaming  arrangements  needed  to  fully  implement  these  tech¬ 
niques  and  still  retain  the  independence  of  oversight  functions  such  as  independent  testing, 
costing,  and  auditing?  In  the  corporate  world,  checks  and  balances  are  built  in  when  estab¬ 
lishing  the  development  team  through  an  appropriate  set  of  management  incentives. 
Whether  this  approach  can  ever  be  fully  implemented  in  the  government  acquisition  envi¬ 
ronment  remains  to  be  determined.  A  key  question  is  whether  appropriate  incentive  mech¬ 
anisms  can  be  designed  and  implemented  to  explore  ways  to  improve  the  organizations 
[Rupp  1993,  pp.  275-330].  The  identification  of  these  mechanisms  should  be  at  the  top  of 
DoD’s  agenda  for  many  reasons  and  not  just  for  incorporating  simulation  in  the  acquisition 
process. 

Functionally  Integrated  Team  Development.  The  second  question  is,  how  to 
organize  a  team’s  work  to  ensure  that  it  follows  an  integrated  development  process  rather 
than  the  traditional  sequential  process?  The  sequential  approach  is  implicitly — if  not 
explicitly — embodied  in  the  DoD  milestone  process,  which  traditionally  has  focused  on 
first  defining  performance  requirements;  second,  validating  the  feasibility  of  performance 
design;  and  third,  developing  the  manufacturing  and  support  infrastructure.  Integrated 
development  implies  that  instead  of  this  sequential  approach,  each  of  these  considerations 
should  be  given  balanced  weight  at  each  point  in  the  development  cycle.  To  do  this,  the 
development  process  needs  to  be  restructured  to  make  it  clear  that  every  function  receives 
appropriate  consideration  at  each  step.  As  an  initial  step,  each  of  the  DoD’s  Advanced 
Technology  Demonstration  (ATD)  and  Advanced  Concept  Technology  Demonstration 
(ACTD)  programs  should  be  designed  to  achieve  a  balanced  set  of  objectives  covering  each 
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of  the  six  functional  areas.  This  also  means  that  each  of  the  six  functional  communities 
must  be  considered  from  the  very  beginning  of  program  definition.  From  that  base,  repre¬ 
sentative  new  system  programs  should  incorporate  such  balance. 

Clear  Communication  and  Expectations.  The  third  question  is,  what  changes  are 
needed  so  that  development  teams  can  establish  clear  lines  of  communication  with  each  of 
the  functional  communities  including  senior  DoD  managers,  oversight  officials,  and  the 
budgeting  processes?  Two  elements  must  be  addressed.  The  first  key  for  successful  com¬ 
munication  is  for  the  development  team  to  clearly  define  standards  of  .  success  for  their  pro¬ 
gram.  In  an  integrated  development  program,  team  members  representing  oversight 
functions  should  join  with  the  other  functional  communities  in  defining  program  perfor¬ 
mance  metrics.  For  example,  both  program  engineers  and  testers  will  understand  what  is 
required  to  successfully  complete  developmental  and  operational  tests.  Such  metrics  would 
serve  as  a  basis  for  communicating  up  and  down  the  management  chain,  as  well  as  across 
functional  boundaries.  Nothing  is  more  corrosive  to  relationships  between  the  develop¬ 
ment  team  and  oversight  officials  than  a  disappointment  resulting  from  unrealistic  program 
expectations.  Programs  based  on  established  technologies  incorporated  into  a  weapon  con¬ 
cept  by  a  cross-functional  team  should  be  more  realistic  than  many  programs  traditionally 
have  been,  and  this  added  realism  fosters  improved  communication. 

Simultaneous  Development  The  fourth  question  is,  how  can  each  of  the  many  dif¬ 
ferent  functional  communities  develop  its  contributions  to  the  design  simultaneously? 
Often  the  best  design  is  very  dependent  upon  which  particular  manufacturing  plants  will  be 
producing  the  various  parts  of  the  system  and  supplying  the  logistics  needs;  the  particular 
location  and  way  the  system  is  going  to  be  tested  (both  development  and  operational);  and 
the  concept  through  which  it  will  be  supported  and  repaired.  Each  production  plant  has  dif¬ 
fering  capabilities  and  significant  cost  savings  can  occur  if  the  design  is  modified  to  accom¬ 
modate  the  capabilities  of  a  specified  manufacturing  site.  Since  testing  is  an  integral  part  of 
the  development  process,  key  data  must  be  readily  obtainable  throughout  the  development 
cycle  in  order  to  assess  the  performance  of  the  system.  Since  each  test  site  has  differing 
capabilities,  the  design  can  be  varied  to  ensure  maximum  compatibility  with  the  selected 
tests  sites.  Life  cycle  costs  are  driven  by  the  operational  and  support  costs.  Options  for 
usage,  not  just  operational  performance,  need  to  be  reviewed  extensively  during  the  design 
phase.  Logistics  and  support  concept  options  also  vary  both  from  a  legacy  aspect  as  well  as 
possible  other  new  systems;  both  need  to  be  an  integral  part  of  the  design  activities. 


To  help  ensure  that  all  of  these  factors  are  properly  included  within  a  design,  all  of 
these  aspects  should  be  considered  nearly  simultaneously  instead  of  sequentially.  This 
would  avoid  costly,  long,  and  drawn  out  iterations  of  the  design.  The  goal  is  many  itera¬ 
tions,  in  a  short  period  of  time,  of  both  basic  design  concepts  and  the  detailed  subsystems. 
By  keeping  a  close  watch  on  the  overall  performance  and  other  associated  goals,  more 
detailed  design  aspects  can  be  developed  collectively  by  key  personnel  from  all  of  the  func¬ 
tional  communities.  Consequently,  each  community  will  need  to  address  functional  and 
interconnection  roles  during  the  design  development  process. 

Risk  Management.  Finally,  the  fifth  question  is,  how  to  structure  programs  to 
ensure  effective  risk  management?  Successful  commercial  developers  have  adopted  the 
TQM  adage  of  “do  it  right  the  first  time”  in  their  development  programs.  An  important  cor¬ 
ollary  of  this  approach  is  the  need  to  reach  design  convergence  before  beginning  detailed 
engineering  and  manufacturing  development.  The  data  on  DoD  program  experience  pre¬ 
sented  earlier  suggests  that  effective  risk  management  could  substantially  reduce  develop¬ 
ment  costs  by  reducing  the  changes  and  rework  needed  in  full-scale  development. 

The  DoD  is  already  placing  greater  emphasis  on  risk  management  in  the  acquisition 
process.  In  particular,  the  feasibility  of  new  technologies  is  now  supposed  to  be  demonstrat¬ 
ed  through  ATD  or  ACDT  programs  prior  to  their  incorporation  in  an  acquisition  program. 
The  recent  DoD  acquisition  strategy  contains,  as  one  element,  an  increased  emphasis  on  the 
use  of  ATDs.  ATDs  are  to  be  used 

...  to  conduct  more  rigorous  ‘up-front’  technology  developments  so  that 
the  acquisition  cycle  can  be  made  less  risky  . . .  TTiese . . .  will  range  from 
demonstrating  the  military  utility  of  new  technological  concepts  in  a  labo¬ 
ratory  environment  to  integrating  and  assessing  technology  in  as  realistic 
environment  as  possible.  [DoD  1992,  p.  1-16] 

But  discipline  and  careful  attention  are  needed  to  make  this  approach  work.  Simu¬ 
lation  technologies  can  play  a  major  role  in  this  because,  as  we  have  seen,  these  technolo¬ 
gies,  used  in  combination  with  traditional  development  processes  (engineering  and 
manufacturing)  and  their  development  tools,  can  provide  a  cost-effective  way  for  the  DoD 
to  identify  risks  early  and  thus  reduce  them  in  the  initial  development  phases. 

The  five  principles  of  these  new  management  practices — strong  leadership,  func¬ 
tionally  integrated  team  development,  clear  communications  and  expectations,  simulta¬ 
neous  development,  and  effective  risk  management — ^provide  the  basic  guidance  needed 
for  improving  defense  development  programs.  As  we  have  shown,  some  of  these  principles 
can  be  implemented  within  the  DoD’s  existing  formal  acquisition  process.  In  particular. 
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there  are  no  formal  barriers  to  improving  communications  and  risk  management  Although 
the  process  does  not  prevent  simultaneous  development,  it  strongly  discourages  the  cre¬ 
ation  of  the  fully  integrated  development  teams  needed  to  implement  simultaneous  devel¬ 
opment.  The  reasons  are  the  legal  and  cultural  barriers  to  teaming  involving  all  six 
functional  communities.  Earlier,  we  posed  the  question,  is  it  possible  to  create  the  kind  of 
teaming  arrangements  needed  to  fully  implement  these  techniques  and  still  retain  the  inde¬ 
pendence  of  oversight  functions  such  as  independent  testing,  costing,  and  auditing?  The 
main  challenge  facing  the  DoD  is  to  provide  a  convincing  affirmative  answer  to  this  ques¬ 
tion. 

C.  SIMULATION  AND  ACQUISITION  REFORM 

We  believe  distributed  simulation’s  ultimate  contribution  will  be  in  supporting 
functional  integration  in  weapon  development  programs.  Nevertheless  simulation  capabil¬ 
ities  also  contribute  to  other  acquisition  process  reform  goals  and  recent  acquisition  process 
reform  proposals.  Two  of  the  main  reform  proposals  under  consideration  are  “prototyping 
plus”  and  “dual  use.”  Prototyping  plus  provides  a  strategy  for  maintaining  military  technol¬ 
ogy  leadership  despite  shrinking  defense  budgets,  by  spending  a  greater  share  of  modern¬ 
ization  resources  on  prototypes  and  limited  production  runs.  This  strategy  focuses  on 
demonstrating  and  testing  leading-edge  technologies,  with  the  idea  that  large  numbers 
could  be  fielded  if  and  when  the  need  arises.  It  is  consistent  with  many  of  the  recent 
attempts  to  define  a  more  flexible  acquisition  process.  These  include  virtual  swords  [Gold 
flexible  acquisition  [Richan  1990,  pp.  \5-\l];  rollover-plus  [As^in  1992]; prototyp¬ 
ing-plus  [OTA  1992,  pp.  12-13, 51-1 5];  and  dual-track  prototyping  [OTA  1989,  pp.  11-13]. 

The  dual  use  acquisition  strategy  is  aimed  at  reducing  the  DoD  reliance  on  defense- 
specialized  suppliers.  This  would  include  generation  of  the  needed  new  and  military-spe¬ 
cific  designs  for  parts  and  components  by  defense-specialized  suppliers.  This  strategy 
would  also  allow  the  DoD  to  tap  the  nation’s  (and  perhaps  the  world's)  broad  commercial 
base,  reducing  acquisition  costs  and  increasing  the  potential  pool  of  suppliers.  We  will 
briefly  outline  the  implications  of  these  strategies  for  the  defense  acquisition  process,  and 
describe  the  role  that  simulation  could  play  in  such  a  process. 

1.  An  Acquisition  Process  for  Flexibility  and  Dual  Use 

A  milestone  structure  that  incorporates  the  flexibility  entailed  by  the  prototyping 
plus  strategy  requires  breaking  the  traditional  acquisition  “pipeline”  by  de-coupling  the 
front  end  of  the  process  from  the  development  and  production  steps  (Figure  IV- 1).  The  front 
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Figure  IV>1.  Acquisition  Reform  and  the  Milestone  Process 


end,  “idea  generation”  process  must  allow  new  ideas  and  proposals  to  chum  indefinitely 
without  the  necessary  detail  planning  that  makes  up  a  full  new  program  start.  The  remain¬ 
ing  steps  in  the  acquisition  “pipeline”  should  be  restructured  to  (1)  incorporate  the  five  prin¬ 
ciples,  and  (2)  shift  the  new  program  start  decision  to  tl;e  beginning  of  detailed  engineering 
and  manufacturing  development.  This  iiamework  would  provide  flexibility,  both  in  the  tim¬ 
ing  of  programs  and  the  magnitude  of  buys.  The  timing  of  new  programs  is  controlled 
through  the  flexibility  in  the  idea  generation  process.  New  technologies  and  manufacturing 
processes  in  a  warfare  area  can  be  created,  prototyped,  and  simulated  indefinitely  without 
a  commitment  to  undertake  production.  The  size  of  buys  can  be  managed  at  the  same  time 
decisions  are  made  on  the  production  concept,  operational  needs,  and  support  capabilities. 

Each  of  the  steps  in  this  functionally  integrated  system  acquisition  process  is  dis¬ 
cussed  in  turn. 


The  first  step  is  idea  generation.  The  underlying  principle  of  “prototyping  plus”  is 
that  the  acquisition  system  should  be  structured  so  that  technologies  can  be  developed  with¬ 
out  necessarily  leading  to  a  new  weapon  program  start.  Hence,  the  idea  generation  process 
must  be  de-coupled  from  the  traditional  acquisition  “pipeline.”  Under  the  pipeline 
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approach,  technologies  were  only  developed  to  meet  a  requirement  for  fielding  a  new  sys¬ 
tem.  Development  activities  were  thus  coupled  with  the  plan  to  field  the  weapon,  and 
scheduled  to  meet  the  associated  system  program  nulestone  requirements.  In  a  de-coupled 
framework,  technologies  would  be  managed  by  warfare  area,  and  would  be  explored 
through  an  ongoing,  level-of-effort  research  and  development  program  coupled  to  the  idea 
generation  process,  without  necessarily  being  linked  to  a  requirement  to  field  a  specific 
weapon. 

At  the  heart  of  the  idea  generation  process  are  ATD  and  ACTD  programs.  ATDs  can 
be  designed  to  serve  a  wide  range  of  purposes.  They  may  focus  on  a  component,  such  as  a 
weapon  sensor;  or  a  subsystem,  such  as  a  ship  propulsion  system;  or  even  a  fully  integrated 
system,  such  as  an  armored  vehicle  or  an  aircraft.  They  may  also  demonstrate  applications 
for  a  basic  material  or  resource.  In  addition  they  could  demonstrate  a  production  process,  a 
test  capability  or  method,  or  focus  on  a  logistics  issue.  ATDs,  when  combined  into  ACTDs, 
could  produce  a  real  prototype,  or  where  simulation  technology  is  sufficiently  advanced, 
they  could  rely  on  a  “virtual  prototype.”  In  parallel  with  these  ACTDs,  concept  studies 
could  examine  the  military  application  and  utility  of  the  demonstrated  systems  in  a  warfare 
area. 

A  typical  ATD  program  may,  for  example,  demonstrate  the  feasibility  of  advanced 
sensors.  Subsystem  prototypes  coupled  with  engineering  and  manufacturing  simulations 
could  be  used  for  examining  the  producibility  of  the  new  design.  Parallel  combat  simula¬ 
tions  might  consider  the  utility  of  this  sensor  in  performing  military  tasks — for  example, 
finding  high-value  mobile  targets  in  a  WAR  BREAKER  type  scenario.  Based  on  these  anal¬ 
yses,  one  of  three  choices  could  be  made: 

•  The  ATD  could  be  allowed  to  lapse,  having  found  the  technology  is  not  present¬ 
ly  useful. 

•  A  follow-on  ATD  could  be  commissioned,  in  order  to  start  a  new  cycle  of  exper¬ 
imentation  and  simulation. 

•  A  decision  could  be  made  to  incorporate  the  technology  in  a  new  ACTD  which 
would  demonstrate  a  weapon  concept  or  a  weapon  upgrade. 

The  idea  generation  process  is  intended  to  provide  a  continuous  cycle  of  experimen¬ 
tation  and  improvements  in  each  of  the  DoD’s  warfare  areas.  Because  ATDs  form  the  foun¬ 
dation  for  next  generation  programs,  it  is  essential  that  a  balanced  program  of  ATDs  be 
maintained.  The  DoD  is  adopting  a  strategic  management  firamework  for  Science  and  Tech- 
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nology  (S«feT)  Thrusts  that  provides  the  needed  perspective  to  create  a  balanced,  diverse 
program  of  ATDs  [DoD  1992]. 

ATDs  should  also  play  a  central  role  in  supporting  the  DoD’s  dual  use  strategy.  The 
DoD  regulations  have  long  stipulated  a  preference  for  off-the-shelf  parts  or  components 
whenever  possible  [DSB  Man  1993].^  Thus  Figure  IV- 1  shows  that  ATDs  might  incorpo¬ 
rate  commercial  as  well  as  defense-specialized  technology.  Despite  these  regulations, 
designers  have  strong  incentives  to  maximize  the  performance  of  weapon  systems,  and 
very  little  incentive  to  incorporate  off-the-shelf  parts  or  components  into  their  systems.  If 
the  DoD  is  serious  about  increasing  the  use  of  commercial  components,  a  stronger  set  of 
incentives  for  developers  will  have  to  be  created.  The  DoD  can  create  the  needed  incentives 
by  commissioning  ATD  projects  with  the  express  purpose  of  adapting  commercial  technol¬ 
ogies  to  military  uses.  For  example,  a  series  of  ATDs  could  examine  how  DoD  navigation 
needs  might  be  met  with  off-the-shelf  global  positioning  unit  technology. 

ATDs  can  also  support  dual  use  strategies  through  their  focus  on  manufacturing 
processes.  Even  when  products  are  specialized  for  defense  use,  they  may  still  be  producible 
with  standard,  commercially  available  production  equipment.  For  example,  many  compo¬ 
nents  of  the  Ml  tank  are  made  in  commercial  job  shops.  Indeed,  most  of  the  machines  used 
for  making  the  M 1  are  found  in  large  numbers  in  factories  throughout  the  United  States. 
The  effort  to  broaden  the  supplier  base  should  not  be  limited  to  requiring  the  use  of  off-the- 
shelf  parts  and  components  where  possible;  it  should  also  include  designing  parts  and  com¬ 
ponents  to  make  use  of  commercially  available  production  tools.  Small  investments  in 
design  could  substantially  increase  the  range  of  producers  capable  of  supplying  parts  and 
components  for  the  next  generation  of  weapons.  The  increased  use  of  CAD/CAM  systems 
and  flexible  manufacturing  will  increase  the  feasibility  of  this  approach  in  coming  years.  It 
is  recognized  that  procurement  regulation  reforms  may  also  be  needed  for  the  dual-use  con¬ 
cept  to  be  effectively  implemented. 

The  second  step  in  the  development  process  is  program  definition.  In  the  envisioned 
framework,  a  functionally  integrated  development  team  would  be  established  when  the 


'  The  DoD's  acquisition  regulations  reflect  the  preference  for  commercial  products.  DoD  regulations  provide 
a  priority  list  of  options  to  the  actual  development  of  a  new  system  that  must  be  considered  prior  to  starting 
a  new  program.  Heading  the  list  are  existing  US.  systems,  followed  by  allied  nation  or  commercial  sys¬ 
tems.  The  regulations  also  require  the  maximum  pr^tical  use  of  commercial  items.  But  such  a  provision 
is  very  difficult  to  enforce  because  a  case  can  always  be  made  that  a  custom-made  component  can  do  a  job 
somewhat  better.  Thus,  the  provision  may  generate  a  new  paperwork  requirement  for  waivers  but  may  have 
little  real  effect  on  development  practices. 
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decision  is  made  to  begin  a  new  program.  Milestone  0  is  retained  in  this  framework,  not  as 
a  hurdle,  but  rather  as  a  notihcation  requirement,  informing  all  DoD  communities  of  an 
organization’s  intention  to  propose  a  new  program.  The  functionally  integrated  team  would 
include,  as  integral  members,  people  from  all  functional  communities  including  indepen¬ 
dent  testing,  costing,  and  audit  organizations.  The  intention  is  to  eliminate  the  traditional 
adversarial  relationship  by  creating  a  team  working  together  to  achieve  common  objec¬ 
tives.  These  individuals  would  help  the  team  craft  program  performance  metrics  that 
include  a  wide  range  of  success  criteria  in  each  community.  The  team  would  establish  the 
basis  for  subsequent  communicadon  up  and  down  the  management  chain  and  across  the 
functional  communities. 

The  dual  use  strategy  should  be  carried  into  the  program  definition  and  subsequent 
phases.  During  these  phases,  the  design  process  can  be  used  to  incorporate  dual  use  tech¬ 
nologies  in  two  ways,  just  as  was  described  for  ATDs  earlier:  first,  by  inducing  designers 
to  use  commercially  available  parts  and  components;  and  second,  by  designing  defense- 
specialized  components  to  be  manufactured  on  commercially  available  manufacturing 
equipment.  These  tactics,  although  not  universally  accepted  by  some  commercial  firms, 
would  substantially  reduce  the  technological  barriers  to  the  integration  of  the  defense  and 
civilian  production  base. 

The  de-coupling  of  the  traditional  acquisition  pipeline  means  that  the  major  pro¬ 
gram  start  decision  would  be  made  later  in  the  process  than  has  traditionally  been  the  case. 
Milestone  n  thus  becomes  the  main  program  decision  point.  At  this  time  a  detailed  program 
proposal  would  be  reviewed  and  approved  by  the  political  leadership.^  The  program  pro¬ 
posal  would  include  much  of  the  same  information  prepared  for  a  Milestone  D  review 
today.  This  would  include  such  items  as  a  COEA  (which  may  rely  heavily  on  distributed 
simulation);  a  test  plan  with  success  criteria;  a  cost,  schedule,  and  risk  management  plan; 
and  a  life-cycle  support  plan.  The  Services  and  OSD  would  form  a  “staffing  team”  to 
review  these  documents,  and  to  raise  questions  when  the  program  is  presented  to  the 
Defense  Acquisition  Board. 


^  In  the  theory  of  concurrent  engineering,  this  decision  should  be  agreed  upon  by  all  responsible  corporate 
officials.  In  government  procurement  this  would  imply  by  analogy  that  milestone  approval  should  be  given 
by  the  sponsoring  Service  or  Agency,  the  Secretary  of  Defense,  the  President,  and  the  Congress.  The 
approval  process  among  these  people  refnesents  an  impwtant  issue  that  must  be  addressed  in  applying  con¬ 
current  engineering  within  the  acquisition  jn'ocess,  but  it  is  beyond  the  scope  of  this  study.  The  reader  is 
referred  to  [DSB  Man  1993]. 
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An  important  component  of  the  Milestone  II  decision  point  is  an  increased  empha¬ 
sis  on  effective  risk  management.  Decision  criteria  for  Milestone  II  should  therefore 
include  a  risk  management  threshold.  This  threshold  can  be  set  relatively  high  because  it 
should  be  possible  to  achieve  design  convergence  prior  to  the  third  step,  full-scale  devel¬ 
opment,  given  that  program  proposals  are  based  on  ATD  and  ACTD  building  blocks.  Of 
course,  there  will  always  be  some  problems  that  emerge  when  a  system  enters  develop¬ 
ment,  but  this  approach  will  reduce  the  problems  that  have  traditionally  been  encountered 
when  a  program  enters  engineering  and  manufacturing  development.  Simulation  used 
throughout  the  process  will  provide  early  learning  to  reduce  risk  and  thus  reduce  cost. 

Since  Milestone  II  approval  for  entering  full  scale  development,  in  practice,  entails 
a  large  commitment  of  funding,  it  should  only  be  done  when  some  level  of  production  is 
anticipated.  The  integrated  development  team  would  have  developed  the  basics  of  the  man¬ 
ufacturing  processes  concurrently  with  the  product  design  prior  to  the  Milestone  II  deci¬ 
sion.  The  goal  of  the  number  of  units,  or  at  least  the  range  of  units,  to  be  produced  must 
have  been  previously  established  so  that  the  scale  of  the  production  system  would  have 
been  known.  There  is,  of  course,  complete  flexibility  in  deciding  on  the  planned  scale  of 
production.  The  level  of  production  agreed  upon  at  Milestone  n  may  be  very  limited  for 
some  programs,  where  the  intent  is  only  to  develop  and  try  out  a  new  weapon  technology. 
Or  a  decision  may  be  made  to  create  a  small-scale  production  system  designed  to  be 
expanded  if  necessary  at  some  future  time.  In  other  cases,  the  intention  may  be  to  build 
enough  to  fully  modernize  the  force.  The  DoD  process  thus  could  accommodate  a  flexible 
approach  to  acquisition. 

This  proposed  process  would  significantly  cut  the  time  required  to  develop  a  weap¬ 
on.  There  are  two  main  reasons  for  this.  First,  much  of  the  time  that  is  often  occupied  with 
the  churning  of  new  ideas  and  sorting  out  preferred  technological  options  at  the  outset  of 
traditional  acquisition  programs  would  be  spent  within  the  idea  generation  process.  Sec¬ 
ondly,  new  programs  would  have  a  firmer  base  of  technological  options  to  build  upon  since 
they  would  be  using  the  ATD  and  ACTD  prototypes  and  simulations.  As  a  result,  the  devel¬ 
opment  of  the  program  concept  should  proceed  relatively  quickly  once  a  decision  to  define 
a  new  program  is  made.  Although  idea  generation  can  proceed  at  measured  pace  in  this  pro¬ 
cess,  a  more  effective  development  process,  combined  with  the  improved  management  of 
risk,  should  significantly  shorten  the  time  required  for  development  and  reduce  cost. 

The  forth  acquisition  step  is  production.  The  program  proposal  reviewed  at  Mile¬ 
stone  n  would  include  criteria  for  Milestone  III  approval  to  enter  production.  Key  in  this 


concept  is  the  inclusion,  within  the  design  iteration  process,  of  the  particular  production 
processes  to  be  used  in  the  noanufacture  of  the  system.^  In  those  cases  where  a  program  pro¬ 
ceeds  more  or  less  according  to  plan,  approval  would  thus  be  pro  forma.  When  programs 
run  into  problems.  Milestone  ID  would  be  a  problem-solving  session  in  which  program 
alternatives  would  be  weighed  in  view  of  the  development  problems  incurred.  Documen¬ 
tation  and  staffing  for  Milestone  III  would  focus  on  the  problems,  and  thus  would  vary  to 
fit  the  circumstances. 

2.  The  Roles  of  Simulation 

Distributed  simulation  provides  a  number  of  capabilities  that  support  this  flexible 
acquisition  process  (Table  IV- 3).  In  the  idea  generation  step,  simulation  serves  to  comple¬ 
ment  and  assess  ATDs  and  ACTDs.  As  technology  is  demonstrated,  simulation  capabilities 
can  explore  possible  applications  and  can  combine  various  technologies  to  explore  system 
of  system  applications.  Many  of  the  examples  discussed  earlier  show  how  the  DoD  is 
already  using  simulation  technologies  to  evaluate  technologies  at  this  early  phase  of  their 
development. 

As  programs  proceed  into  program  definition  and  development,  distributed  simula¬ 
tion  will  act  as  a  tool  supporting  functionally  integrated  development.  The  simulation  capa¬ 
bilities  of  visualization,  performance  modeling,  common  data,  and  user  interaction  apply 
in  these  phases  in  the  ways  discussed  earlier.  One  additional  application  that  was  not 
emphasized  earlier  is  the  use  of  simulation  in  communicating  program  parameters  to  over¬ 
sight  authorities  at  the  Milestone  n  decision  point.  This  could  make  a  significant  contribu¬ 
tion  to  the  understanding  of  the  program  and  the  effective  management  of  risk. 

Acquisition  reform  and  simulation  technology  improvements  will  interact  synergis- 
tically  to  improve  the  acquisition  process.  The  full  use  of  simulation  directly  supports  func¬ 
tionally  integrated  development  which  should  be  a  key  feature  of  any  acquisition  reform. 
Yet  acquisition  reforms  will  set  in  motion  the  need  to  more  enhanced  understanding  among 
all  parties  involved  within  acquisition  and  such  needs  will  spur  the  incorporation  of  simu¬ 
lation  to  aid  in  that  understanding.  We  find  that  although  some  of  the  principles  of  these 
management  practices  can  be  implemented  within  the  current  formal  acquisition  system, 
some  changes  and/or  reforms  are  needed  to  create  fully  integrated  development  teams. 


^  Not  all  production  facilities  have  the  same  capabilities.  The  “final”  design  should  not  be  derived  until  the 
actual  production  facility  has  been  selected.  The  processes  unique  to  that  facility  need  to  be  included  within 
the  design  and  those  capabilities  discussed  within  the  Milestone  n  review. 
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Table  IV-4.  The  Role  of  Simulation  in  a  Flexible  Acquisition  Process 

Program  Step 

Roles  of  Simulation 

Idea  Generation 

•  A  virtual  prototype  to  substitute  for  real  proto¬ 
types  in  some  ATDs 

•  A  combat  worth  assessment  tool  which  can 
incorporate  human  factors  and  tactical  innova¬ 
tions 

•  A  tool  for  evaluating  system-of-systems  ideas 

•  An  engineering  and  manufacturing  analytical 
tool  to  complement  real  experimentation 

•  Dual  use:  a  communication  tool  for  identifying 
commercially  available  parts,  components,  and 
processes 

Program  Definition 

•  A  communications  tool  for  functionally  inte¬ 
grated  teams 

•  Analytical  tools  for  use  by  each  of  the  function¬ 
al  communities 

•  Program  visualization  for  Milestone  II  review¬ 
ers,  committees,  and  DAB  members 

•  Dual  use:  a  corrununication  tool  for  identifying 
commercially  available  parts,  components,  and 
processes 

Full  Scale  Development 

•  Provides  early  learning  for  risk  reduction  and 
cost  avoidance 

•  A  communications  tool  for  functionally  inte¬ 
grated  teams 

•  Analytical  tools  for  use  by  each  of  the  function¬ 
al  communities 

•  Dual  use:  a  communication  tool  for  identifying 
commercially  available  parts,  components,  and 
processes 

Production 

•  Tool  for  assessing  alternative  manufacture  pro¬ 
cesses  when  needs  change 

•  Production  process  simulations  to  optimize 
manufacture  of  multiple  products  on  same  lines 
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V.  FINDINGS  AND  CONCLUSIONS 


The  commercial  experience  cited  at  the  outset  of  this  paper  provides  benchmarks 
for  estimating  the  gains  possible  in  the  DoD’s  development  programs.  It  also  pinpoints  two 
fundamental  changes  needed  to  apply  the  new  management  principles  to  defense  programs. 
First  is  to  break  down  the  traditional  development  “stovepipes”  or  barriers  between  the 
functional  communities.  Development  teams  should  integrate  across  the  functional  com¬ 
munities  of  requirements  and  concept  development,  engineering  design  and  manufactur¬ 
ing,  test  and  evaluation,  logistics  and  support,  force  planning  and  budgeting,  and  program 
management  Second  is  to  manage  risk  more  effectively.  Now  that  the  Cold  War  is  over, 
programs  should  not  start  full-scale  development  before  major  technological,  operational, 
and  support  problems  are  resolved. 

In  addition  to  these  management  improvements,  the  leading  commercial  firms  are 
also  aggressively  exploiting  simulation-related  development  and  manufacturing  technolo¬ 
gies.  The  role  of  simulation  is  suggested  by  the  four  capabilities  it  offers.  The  first  is  visu¬ 
alization.  Visualization  may  be  static  or  dynamic,  but  it  permits  all  who  view  an  item  to  far 
better  understand  its  design  and  function,  allowing  problems  to  be  surfaced  before  hard¬ 
ware  is  built.  The  second  capability  is  broad-based  performance  modeling.  This  could  in¬ 
clude  simulating  physical  performance  such  as  mechanical  stress  or  a  weapon  in  combat. 
This  gives  users  and  developers  a  more  complete  understanding  of  how  a  component  or 
system  would  work  in  the  real  world.  The  third  capability  is  a  common  information  base. 
It  gives  all  team  members  up-to-date  data  on  designs,  and  facilitates  rapid  iterations  in  de¬ 
sign  and  associated  processes  as  problems  are  surfaced.  Finally,  the  fourth  capability  is  user 
interaction.  This  could  range  from  manufacturers  marking  up  drawings  firom  locations  re¬ 
mote  from  the  designers,  to  test  pilots  participating  in  simulated  air-to-air  combat,  to  the 
interaction  between  logistics  planners  and  system  designers. 

Simulation  technologies  providing  these  capabilities  provide  a  powerful  tool  for 
implementing  a  functionally  integrated  acquisition  process.  Visualization,  performance 
modeling,  and  user  interaction  assist  communication  both  within  and  across  functions,  thus 
breaking  down  stovepipes.  Performance  modeling,  cormnon  information,  and  user  interac- 


tion  all  assist  analysis  and  should  significantly  improve  the  management  of  risk. 

The  degree  to  which  simulation  can  actually  provide  these  capabilities — both  today 
and  in  the  future — depends  on  the  power  and  acceptance  of  the  available  simulation  tools. 
Our  examples  show  there  are  valuable  applications  of  simulation  within  each  of  the  DoD’s 
functional  communities.  But  as  yet,  there  is  little  success  in  integrating  simulation  across 
functional  boundaries.  Currentiy,  most  simulations  fall  into  four  separate  families  of  simu¬ 
lation:  combat,  physical,  availability  and  support,  and  cost  and  schedule.  While  combat 
simulation  may  get  the  most  attention,  less  glamorous  but  cost-effective  applications  have 
been  made  in  each  of  the  functional  communities.  We  find,  however,  that  most  existing 
simulation  tools  have  been  tailored  to  the  needs  of  the  communities  that  use  them,  so  they 
have  not  been  designed  to  communicate  across  functions.  Consequently,  the  best  available 
simulation  practices  are  not  uniformly  applied  across  programs. 

When  simulation  capabilities  are  fully  matured,  distributed  simulations  will  elec¬ 
tronically  connect  together  the  system  design  representations  and  the  analysis  tools  of  all 
acquisition  disciplines  and  functions.  They  will  involve  the  many  different  internal  DoD 
acquisition-related  functions  and  organizations  and  the  associated  defense  industries.  They 
will  provide  a  more  effective  way  to  participate  in  the  iteration  and  refinement  of  system 
designs.  These  iterations  will  take  place  long  before  the  first  piece  of  production  metal  is 
cut  or  before  the  first  line  of  operational  computer  code  is  written,  and  will  be  used  in  the 
acquisition  of  both  new  and  re-designed  systems.  Key  personnel  within  each  of  those  func¬ 
tional  communities  will  be  able  to: 

•  More  easily  assess  the  system’s  cost  effectiveness  within  proposed  environ¬ 
ments  and  scenarios  from  the  perspective  of  their  own  discipline;  through  this 
tool  they  will  able  to  extend  the  utilization  of  their  own  discipline’s  analysis 
tools.  Through  the  use  of  simulation,  they  will  be  immersed  in  the  system  oper¬ 
ation  as  it  is  viewed  from  within  their  own  functional  discipline.  This  will  pro¬ 
vide  a  greater  understanding  of  the  system’s  operation  and  effectiveness  within 
other  disciplines.  These  key  personnel  will  be  able  to  directly  transfer  the 
designs,  environments,  and  scenarios  into  their  own  functional  models  and  sim¬ 
ulations  electronically,  and  thus  avoid  many  or  all  of  the  data  input  errors.  They 
can  conduct  analysis  on  those  designs,  environments,  and  scenarios  using  their 
own  detailed  and  validated,  verified,  and  accredited  models  and  simulations. 

•  Communicate  potential  problems  in  the  designs,  environments,  and/or  scenari¬ 
os,  and  offer  suggestions  or  other  options.  Team  members  will  be  able  to  express 
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their  concerns  and  suggestions  into  a  common  electronic  medium.  Their  com¬ 
ments  will  be  sent,  along  with  the  representations,  directly  to  the  designers  and 
responsible  system  integrators. 

•  Accept  and  collectively  validate  an  optimized  system  design  and  the  associated 
environments  and  scenarios.  As  integrated  team  members,  these  key  personnel 
will  be  able  to  consider  all  concerns  and  suggestions  raised  by  the  various  dis¬ 
ciplines  and  help  reach  an  understanding  of  and  an  overall  rationale  for  the 
designs,  environments,  and  scenarios.  By  “signing  off”  on  each  major  iteration 
of  the  design  and  the  associated  environments  and  scenarios,  they  will  be  giving 
their  approval  to  the  design. 

Distributed  simulations,  when  applied  in  a  functionally  integrated  system  acquisi¬ 
tion  process,  will  capitalize  on  expanding  information  technology,  complement  the  numer¬ 
ous  new  acquisition  enhancement  approaches,  and  will  incorporate  the  existing  extensive 
modeling  and  simulation  capabilities  within  each  of  the  functional  communities.  Since 
most  of  the  key  personnel  within  those  communities  are  already  geographically  dispersed 
and  yet  need  to  be  actively  involved  in  each  phase  or  iteration  of  the  acquisition  process, 
distributed  simulation  offers  a  very  effective  means  to  collect  the  ideas  of  those  key  people. 
This  concept  also  permits: 

•  Considering  the  latest  technologies  for  incorporation  within  potential  new  sys¬ 
tems  or  re-designed  systems. 

•  Anticipating,  as  best  as  possible,  major  costly  errors  so  they  can  be  avoided. 

•  Developing  the  best  designs  that  can  be  ready  for  production  when  deemed 
appropriate. 

It  also  supports  evaluating  the  designs  in  diverse  environments,  in  differing  scenarios,  and 
with  optional  mixes  of  systems  with  which  this  system  may  need  to  operate. 

We  have  described  six  main  building  blocks  that  should  compose  an  investment 
strategy  for  simulation,  i.e.,  simulation  hardware  and  software;  application  tools;  commu¬ 
nication  standards  and  protocols;  behavioral  models  and  data;  verification,  validation  and 
accreditation;  and  education  and  training.  TTiese  building  blocks  form  a  general  outline  that 
can  be  fashioned  into  a  fully  developed  strategy  by  identifying  specific  projects  within  each 
area  and  appropriately  balancing  resources  across  areas.  The  priorities  within  this  strategy 
should  reflect  the  current  state  of  the  art  and  ongoing  development  activities  in  each  block, 
and  allocate  resources  appropriately  to  achieve  balance  in  progress  across  capabilities.  In 
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doing  this,  the  DoD  should  take  advantage  of  ongoing  commercial  simulation  technologies 
because  the  commercial  sector  is  leading  in  the  development  of  engineering  simulation  and 
manufacturing  tools,  as  well  as  in  the  development  of  simulation  and  networking  hardware. 

It  should  also  be  recognized  that  many  of  the  functional  communities  will  continue  to  de¬ 
velop  their  own  capabilities  to  meet  their  own  requirements. 

The  DoD’s  acquisition  community  can  most  effectively  emphasize  initiatives  and 
investments  in  those  areas  that  will  not  otherwise  be  supported.  These  initiatives  include 
the  following: 

a.  Design  an  acquisition  simulation  architecture  that  encompasses  all  of  the  acqui¬ 
sition  communities  and  families  of  simulations  and  provides  bridges  between 
them.  This  architecture  should: 

1.  Describe  interfaces  between  the  simulations  used  within  each  type  of  simu¬ 
lation,  simulation  family,  and  functional  community. 

2.  Define  subarchitecture  options  which  may  be  most  useful  within  each  com¬ 
munity,  yet  be  interconnectable  by  means  of  an  overarching  architecture  and 
thereby  to  those  within  the  other  communities. 

3.  Develop  variable  resolution  simulation  capabilities  within  simulation  fami¬ 
lies  as  required  by  each  functional  community. 

b.  Create  a  user  guide  for  simulation  in  acquisition  that  describes  best  practice 

applications  of  modeling  and  simulation  within  and  across  functional  commu¬ 
nities,  along  with  lessons  learned.  This  guide  should:  ( 

1 .  Provide  an  overview  text  for  program  managers — incorporate  this  within 
the  Defense  Systems  Management  College  curriculum. 

2.  Inform  all  personnel  within  the  acquisition  discipline  of  the  benefits,  limita¬ 
tions,  and  approved  use  of  distributed  simulation. 

c.  Develop  new  cost  and  schedule  simulation  tools  that  show  how  advances  in 
management  practices  and  developing  technologies  can  be  incorporated  in 
structuring  new  programs.  These  tools  would: 

1.  Expand  the  use  of  Activity-Based  Costing  (ABC)  within  programs. 

2.  Incorporate  Material  Resource  Planning  data  directly  into  cost  models. 
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d.  Develop  the  means  to  directly  interconnect  the  cost  analysis  parameters  with  the 
detail  design,  manufacmring  processes,  logistics  concepts,  testing  and  evalua¬ 
tion  options,  and  other  key  cost  driving  functional  activities. 

e.  Develop  an  appropriate  distributed  simulation  security  environment  so  that 
classified,  proprietary,  and  personnel  sensitive  data  can  be  readily  exchanged 
among  authorized  simulations  and  users. 

f.  Invest  in  the  infrastructure  needed  to  apply  simulation  in  acquisition  programs. 
Investments  should  be  designed  to: 

1.  Build  accessible  data  bases  that  capture  prior  experience  and  help  prevent 
reoccurrence  of  mistakes. 

-  Develop  ways  to  obtain  field  training  data  to  help  acquisition  programs 
better  understand  the  capabilities  of  soldiers  in  the  field. 

-  Record  lessons  learned  from  previous  acquisition  programs. 

-  Catalog  contractor-delivered  concept  designs  and  engineering  data  so 
follow-on  programs  do  not  need  to  “start-over.” 

2.  Improve  simulation  development  tools,  including  tools  for  W&A,  variable 
fidelity,  and  aggregation/deaggregation. 

-  Develop  a  capability  maturity  model  for  evaluating  organizations  that 
develop  and  use  models  and  simulations.  This  would  be  similar  to  capa¬ 
bility  models  that  are  used  to  help  assess  the  capabilities  of  software 
developers. 

-  Develop  measures  of  fidelity  to  be  used  scoping  W&A  activities. 

-  Develop  logical  and  mathematical  ba.ses  for  proper  aggregation  and 
deaggregation  of  enities  within  and  among  simulations. 

3.  Establish  widely  acceptable  standards  by  which  the  models  and  simulations 
can  be  developed,  including  data  exchange  standards  to  facilitate  exchanges 
within  and  among  the  functional  communities. 

In  addition  to  these  initiatives  that  mostly  center  on  activities  of  the  acquisition 
functional  communities,  we  also  feel  that  appropriate  incentives  need  to  be  defined  for  or¬ 
ganizations  both  within  and  supporting  the  DoD  to  help  encourage  them  to  adopt  the  key 
management  principles  which  commercial  industries  have  found  necessary  to  attain  an 
functionally  integrated  acquisition/development  process.  These  principles  are: 


•  Selecting  strong  program  development  leadership. 

•  Using  functionally  integrated  development  teams. 

•  Establishing  clear  communications  and  expectations. 

•  Conducting  simultaneous  development. 

•  Implementing  effective  risk  management. 

This  discussion  of  simulation  building  blocks  provides  a  starting  point  for  creating 
an  investment  strategy.  The  vision  of  simulation  as  a  tool  supporting  a  functionally  integrat¬ 
ed  system  acquisition  process  provides  a  guiding  principle  for  targeting  these  investments. 
But  this  vision  is  only  a  point  of  departure — a  fully  developed  strategy  for  simulation  will 
require  an  extensive  effort  from  DoD  leadership  with  help  in  the  adoption  of  the  manage¬ 
ment  practices  and  the  support  of  each  of  the  functional  communities  to  address  the  tech¬ 
nical  issues.  The  creation  of  such  a  strategy  properly  belongs  in  the  hands  of  the 
communities  that  develop  and  use  simulation. 
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LIST  OF  ACRONYMS 


ABC 

Activity-Based  Costing 

ACTD 

Advanced  Concept  Technology  Demonstration 

ADATS 

Air  Defense  Anti-Tank  System 

AMRAAM 

Advanced  Medium  Range  Air-to-Air  Missile 

ARPA 

Advanced  Research  Projects  Agency 

ATACMS 

Army  Tactical  Missile  System 

ATD 

Advanced  Technology  Demonstration 

CAD 

Computer-Aided  Design 

CAE 

Computer-Aided  Engineering 

CALS 

Continuous  Acquisition  and  Life-cycle  Support  (previously  known 
as  Computer-Aided  Acquisition  and  Logistics  Support) 

CAM 

Computer-Aided  Manufacturing 

CAM-I 

Computer-Aided  Manufacturing-International 

COEA 

Cost  and  Operational  Effectiveness  Analysis 

C3I 

Command,  Control,  Communications,  and  Intelligence 

DAB 

Defense  Acquisition  Board 

DICE 

Defense  Advanced  Research  Projects  Agency  (DARPA)  Initiative 
on  Concurrent  Engineering 

DIS 

Distributed  Interactive  Simulation 

DoD 

Department  of  Defense 

DMSO 

Defense  Modeling  and  Simulation  Office 

DSI 

Defense  Simulation  Internet 

EDI 

Electronic  Data  Interchange 

EMD 

Engineering  and  Manufacturing  Development 

Acronyms- 1 


EXCIMS 

F-SAT 

FAADS 

FACS 

FOFA 

FOG-M 

ICAM 

IDA 

IDEF 

IHPTET 

IPPD 

LCM 

MIDS 

MIT 

MUAV 

NASA 

NLOS 

OSD 

OT&E 

PDES 

PERT 

PM 

RAMCAD 

SAFOR 

SIMNET 

TACCSF 

TACWAR 


( 


Executive  Council  for  Modeling  and  Simulations 
Frugal  Satellite 

( 

Forward  Area  Air  Defense  System 
Force  Acquisition  Cost  System 
Follow-on-Forces  Attack 

Fiber  Optic  Guided  Missile  * 

Integrated  Computer-Aided  Manufacturing 
Institute  for  Defense  Analyses 

Integrated  Definition  Language  (formerly  Integrated  Computer-  , 

Aided  Manufacturing  (ICAM)  Definition  language) 

Integrated,  High  Performance  Turbine  Engine  Technology 
Integrated  Product  and  Process  Development 

( 

Logistics  Capability  Model 

Multi-Functional  Information  Distribution  Systems 

Massachusetts  Institute  of  Technology 

Maritime  Unmanned  Aerial  Vehicle  ' 

National  Aeronautics  and  Space  Administration 
Non-line  of  Sight 

Office  of  the  Secretary  of  Defense  , 

Operational  Test  and  Evaluation 

Product  Data  Exchange  Standard 

Project  Evaluation  and  Review  Technique 

Program  Manager 

Reliability,  Availability,  and  Maintainability  in  Computer-Aided 
Design 

Semi-Automated  Forces 
Simulator  Network 

Theater  Air  Command  and  Control  Simulation  Facility 
Tactical  Warfare  (model) 


Acronyms-2 


TQM 

U.S. 

VV&A 

VTOL-UAV 


Total  Quality  Management 
United  States 

Verification,  Validation,  and  Accreditation 

Vertical  Take-off  and  Landing-Unmanned  Aerial  Vehicle 


Acronyms-3 


